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ABSTRACT 


This thesis investigates the relationship between 
Washington based Project Managers and a Test and Evaluation 
(TEE) facility, the Pacific Missile Test Center (PMTC). Five 
areas of the relationship are examined. First, the formal 
weapon system acquisition process as prescribed by current 
directives is presented aS an overview. Next, the Pacific 
Missile Test Center organization is analyzed. This is 
followed by an investigation of funding techniques, and 
Specifically, those employed by PMTC. The Project Managers! 
perception of their participation in the T&E process is 
presented through results of a questionnaire. Finally, an 
objective method of selecting the appropriate TSE facility 


for a project based on technical capability is developed. 
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I. INTRODUCTION 

The Grit metas previous educational background in 
Aeronautical Engineering combined with their current 
education in Administrative Sciences stimulated interest in 
the Weapons System Acquisition Management (WSAli) field. It 
was hoped that thesis research in this area would benefit 
the authors in future billet assignments in this area. In 
order to focus on a specific topic, a literature search of 
current research in this area was conducted. The search 
indicated a number of interesting problems involving the 
Test and Evaluation (I5E) aspects of WSAM. Tt was 
considered desirable to concentrate on the focal point of 
T&E, namely, the Washington D.C. arena. However, due to 
lack of funding support, trips to the Washington area were 
considered impractical. Consequently, problems involving 
TEE activities in the local region were investigated. 
Contacts were made at both the Naval Weapon Center (NWC), 
China Lake, and the Pacific Missile Test Center (PMTC), 


Point Mugu. 


A trip was made to point Mugu for familarization and 
probiem definition purposes. Personnel at Point Mugu were 
very receptive and interested in cooperating with thesis 
research involving PTC. After the initial trip, it was 
decided to concentrate on analyzing the T&E facility/Project 
Manager relationship. Additional trips made to Point Mugu 
focused on understanding the activity's involvement in the 


T&E process. 


The next five chapters represent the results of this 
research with the first four chapters concentrating each on 
a separate aspect of the relationship. First, the formal 
weapon system acquisition process (principly the TSE 
aspects) as prescribed by current directives is presented 


for purposes of providing an overview of how the process 


14 





Should work. Next, a T&E activity's (PMTC) organizational 
development is described together with how the organization 
interacts with its external environment.  Presupposing that 
"he who has the gold rules", the financial management of the 
facility is investigated. This is followed by a different 
viewpoint of the relationship: the Project Manager's 
perception of his participation in the T&E process. The 
Project Manager's opinions were ascertained by means of a 
guestionnaire. Results of the questionnaire provide the 
data for Chapter VI. This chapter proposes an objective 
method of selecting the appropriate T&E facility for a 
particular weapon system's ТЕБЕ requirements based on 


techaical capabilities. 


15 





II. IEST AND EVALUATION WITHIN THE NAVY 


À. ‘ ORGANIZATION FOR RDT&E 


Reference 1 provides a general overview of the Navy 
RDT&E organization and process. Much of the following 
information has been extracted from it in order to briefly 


acquaint the reader with pertinent background information. 


In the National Security Act of 1947 Congress’ tasked 


the Secretary of Defense to : 


"L..eliminate unnecessary duplication in the Department 
of Defense and particularly in the field of research and 


engineering..." 


Later, the Derense Reorganization Act "of 71959 
established the Director of Defense Research and Engineering 
(DDRSE) as the highest ranking assistant secretary of 
defense. The primary responsibilities of DDR&E are to act as 
principal advisor to the Secretary of Defense (SECDEF) on 
Scientific and technical matters, supervise research and 
engineering, and direct and control research and engineering 
activities. To fulfill his responsibilities he has been 
delegated authority to approve, modify, or disapprove 
programs and projects of the Military Departments. DODDIR 
5100.1 (Ref. 2) established the Deputy Director(Research and 
Advanced Technology) (R&AT) and the Deputy Director(Test and 


Evaluation) (DDT&E) as advisors to the Director. 


In addition to the above DOD organization, the Joint 
Chiefs of Staff (JCS) has the responsibility of advising 
SECDEF on matters of strategic guidance, overall military 


requirements, and recommendations of new weapon systems. 
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Figure 1 depicts the Department of the Navy 
Organization for RDT&E. While the overall] responsibility for 
policies and control of the Department of the Navy rests 
with the Secretary of the Navy. Matters related to research, 
development, engineering, test and evaluation are the 
specific responsibility of the Assistant Secretary of the 
Navy (RED) (ASN(RED)) as directed by SECNAVINST 5430.67 (Ref. 
3). He and his small staff act as the principal contact for 
DDREE and manage the "Research, Development, Test and 
Evaluation, Navy" appropriations account. Due to his limited 
staff size he is dependent on advisors such as Director 
RDT&E (OP-098), the Chief of Naval Development (CND), Chief 
of Naval Research (CNR), and the Director of Navy 


Laboratories (DNI). 


The Office of the Chief of Natal Operations 15 
primarily concerned with defining potentially attainable 
Capabilities, appraising the military worth of RDT&EE 
advancements, and approving projects which promise the 
greatest return on invested resources. These policies аге 
Specifically implemented by DRDT&E (0P-098) who acts as the 
focal point for T&E policy. 


The "how" aspects of Navy RDT&E have been delegated to 
the Chief of Naval Material (CNM). His responsibilities 


include: 


1. Translating operational requirements into 


hardware systems and technology objectives. 
2. Managing the technology base. 
3. Defining science and technology capabilities. 


4. Developing detailed plans for RDT&E projects. 


17 





FICY UOT NOLIVZINVOUO XAVX ЯНЬ 50 LUuVdaG "I india ено нн 


-—— == m —. -- —— 


Gh vri O0? N19 3 3NI19N3 
$3!1! Y12V у ЗУЛУМ 





p————- 


з наты Юму зоб | 


— 








3910b 
9137919 


Г-ТЕ ы-ы О! 
3 93530 | 
à Jiv*$ 110 CAV s 

1903700426 WAN ! 


| 32323” 1 
- — = = . o — à 


83110914440? 








| 1 

Kr SOT CN GNA, 
TV] AS et а Аз, м 30 | | 
АЛЕ ГА 


1,55 < | | 


| IYS Yar | 


135N^02 
атттоь | WU IN ID IMA 
ФАРС | 30 371349 


ОМУМЫ32 5У%315А5 
AlddNS IvAVN 











341 40 39393930 





V aN 38 уме Twi? 
NYN' 3I AAVN 39H14 JO 
АНТТЫ 35 INVISISSY 


3^A!'1 vV31519311 
30 391310 : 





(QN vWwiWO) SW31SAS 
21!NOU 412313 IVAYN 


AU 3oun5 ONY 
JNIMGIW 
4N NWVIHNGA е 





5М011Ұ 8340 IWAYN . 
30 $31H9 321Л 


SNOIAYH3dO IVAYN 
30 3314) 


SOuvOS 
VINNOSH 1d 
Зулум 


1N3A30vNYW 
U 340aNvW 
NVIM^!2 30 321330 





I68lv 44v 3^H1S513U ONY 
Й МЭУ ОЈМУИ! ДЛУМ 3мі 30 
АЯУ139235 INVISISSY 


AAWN Эн) 30 
АУУ 135935 YIONN 


591Ұ33ҰҮ 


AAWN 3H) 30 
AuvY13u23S 


om m —- 


ONYVWNWO. SWN31 
SAS Y 35 IVAVN 





(4 N3^34013^30) 


ЗУ 31V l VAVN 
ЗО 33142 А170430 


JNJ Mr Auer Hp t 0м, базу YO, 


UNVAAODI SAL 
SAS wiv TVAVN 





Sr ri DO dd 
AUCIvUOSvA 
30 80159380 





1ViB 31 VN 
ЗУЛУМ 
JO +3149 е 


1375814 
IVAVN 
sO NwWIWNG е 


159009) 5509 ЭмімҰ 
33У15 30 33142 A10 5C 





$4802 3NIU YN 
30 INvVONWAWOD 


S31HO1vdOR v3 
AAWN 30 80123810 








) %%0мі 5інО2 3МіНҰ 
33У15 30 33ІН2 А10430 





H NO 


H2UY3S3U ЗУЛУМ 


30 133142 








$3111^!112v O3314 


H25v 35 3u urn 
10 31410 


длуч ML 30 
d 34V 8^7)^ vv 1350 





4 ^36^«013^ 23 
IVAYN 99 3312 





3910% 
5012350 





1М194033 А30 ONY 
HOuv 15 1Hl AAvVN 34 30 
АНУ; 3423$ INWISISSY 


мо Умно змі De 


30 391130 3ні 30 321330 





31Ұ2САОУ 320v - 


INInLevd3}0 АЛУМ 


321430 
3^'uvw 1SiN IN(Y 








5. Overseeing implementation of RDT&E programs. 
} 

CNM is assisted in the implementation of the above by 
the Deputy Chief of Naval Material (Development) (DCNM(D) ) 
who also acts as Chief of Naval Development (СМО) and 
reports directly to ASN(R5ED). 


The Naval Material Command consists of NAVMAT 
designated Project Managers (PMS) plus the various Systens 
Commands as depicted in Figure 2. NAVMAT designated PMs are 
utilized for projects with critical interfaces between 
Military Services and/or Systems Commands. FOr other 
projects, one of the Systems Commands is placed in charge 


and Projects Managers are deSignated within it. 


NAVAIRINST 5451.80 (Ref. 4) established the Project 
Management Office(AIR) within NAVAIR Headquarters to assist 
in the planning, directing, and controlling general 
Management aspects of specific weapons system projects. As 
Such, they are responsible for maintaining the NAVAIR 
designated project Charters (5530 instruction series), 
communicating broad policy guidance to PMs from higher 
authority, and providing other assistance to PMS as 


required. 


The project management organization is structured by 
purpose and thus cuts across various functional organization 
lines. Ps are generally Navy Captains or Flag Officers 
available for at least a three year tour. OPNAVINST 3960.8 
(Ref. 5) charges the PM with the responsibility for 
coordination and execution of a T&E program responsive to 
the policies expressed in SECNAVINST 5000.1 (Ref. 6). 
Specifically, he must coordinate а test program with the CNO 
development coordinator suitable to answer operational 
questions, prepare a Test and Evaluation Master Plan (TEMP), 


engen or tke performance of TEE and TEE support at 
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appropriate laboratories, contractor facilities ог Т&Е 
activities, and determine the need for assistance of 


operating forces. 


Laboratories responsible for R&D of aircraft and 
missile systems report directly to the Director of 
Laboratory Programs (DLP) in the staff of DCNM(D). The DLP, 
also acting as DNL, is the principal advisor to ASN(R&D) in 
matters of the RDT&E field complex. 


Test and Evaluation of aircraft and aircraft weapon 
systems is generally divided into Operational Test & 
Evaluation (OT&E) and Development Test & Evaluation (DTOE). 
OTEE is the responsibility of the VX sguadrons of the 
Operational Test and Evaluation Forces (OPTEVFOR) which 
report to CNO via OP-983, the Test and Fvaluation Division 
of OP-098. DT&E 1s generally under the cognizance of field 
activities such as the Naval Air Test Center and Pacific 
Missile Test Center which report to NAVAIR. The coordination 
of TEE activity within NAVAIR is currently:delegated tc the 
Test and Evaluation Coordinator (TECO) who is double-hatted 
as CO NATC as directed by NAVAIRINST 5400.27B (Ref. 7). 


Some of the specific tasks of TECO are: provide to 
NAVAIR recommendations and proposed directives concerning 
ШОЕ росу, act as focal point for TSE matters, assist in 
preparation of and evaluate planning documents containing 
TEE requirements, provide. comments to NAVAIR on adequacy of 
TSE plans, review and evaluate workload asSignment to field 
activities, monitor actual T&E accomplishment in order to 
recommend readjustment of available resources, provide 
planning information to field activities, and assist NAVAIR 
PMS in the coordination with COMOPTEVFOR. 


NAVAIR NOTICE STOO ОТЕ” March, 1975 (Ref. 8) 


established the Assistant Commander for Test Evaluation 
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(AIR-06) and called for an AIR-06 operational status by 1 
July 1975. He is responsible for the policy direction, 
planning, management and support of  NAVAIRSYSCOM TSE 
programs, facilities, ranges, and targets. He is the primary 
NAVMAT contact with contractors and other government 
agencies for matters concerning T&E facilities, ranges, and 
targets including all Navy elements of the Department of 
Defense Major Ranges and Test Facilities Base. Specific 
duties include; assisting Project Managers in preparing for 
Defense Systems Acquisition Review Council (DSARC) reviews 
and in budget preparation, providing life-cycle management 
of targets and T&E support systems, providing guidelines for 
and approving mission functions, organization and staffing 


of T&E field activities and ranges, establishing the policy 


‘for the assignment of workload to the T&E field activities, 


establishing priorities for the use of T&E activities, and 
maintaining liaison with COMOPTEVFOR to assure a coordinated 
DT&E and OT&E of NAVAIR weapon systems and equipment. The 
presently anticipated organization within AIr-06 calis for 
an AIR-06T (double-hatted as CO NATC) who would be 
responsible for T&E efforts and AIR-06R (double-hatted as CO 


PMTC) who would be responsible for ali ranges and targets. 


B. PLANNING PROCESS 


The planning system leading to OPNAV-sSupported RDTSE. 
weapon systems has recentiy been revised from the oid system 
in effect since 1961. This paper will address the new 
system; however, projects approved under the old system are 
still in development and remain under the documents produced 


by that systen. 
Figure 3 depicts the documentation and review procedure 


utilized throughout the planning phases of a project. 


Proposals of future concepts, called Advanced Systen 
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23 





Concepts (ASC), are contained in the Navy Advanced Concepts 
(NAC). Up to 30 proposals may be submitted by each Systens 
Command and essentially bridge the gap between science and 
technology and a budgeted project. The Director of  RDTSE 
within the Navy selects a number of these during the Navy 
budgeting cycle (POM Process) for initiation of Advanced 


Development. 


Concise statements of operational needs for 0- 10 years 
are reflected in Operational Requirements (OR). These three 
page requests for proposal from CNM are originated within 


OPNAV at the request of Fleet activities or other commands. 


After technical studies, CNM replies to OPNAV with 
alternative solutions contained in Development Proposals 
(DPs). OPNAVINST 5000.42 (Ref. 9) suggests that an 
iterative process should be developed between the OPNAV OR 
Sponsor and the CNM to prepare the DP and resoive all 


relevant questions. 


In response to the DP, OPNAV prepares a Navy 
Development Concept Paper  (NDCP) for approval by the 
Acquisition Review Committee (ARC), a sub-committee of the 
CNO Executive Board (СЕЗ). The NDCP represents the first 
commitment of OPNAV RDTSE resources. It includes issues, 
objectives, plans, parameters, areas of risk, and 
development alternatives, and provides the basis for a 
Development Concept Paper (DER a: The DCP, as stated in 
OPNAVINST 3930.88 (Ref. 10) is the basic source of advance 


evaluation planning information. 


Although the DCP and DSARC process is utilized only for 
majoc programs, similar processes are used for minor 
programs. A major program is one having an estimated RDIG& 
COS ШО с с=с ОГЫ million, an estimated production cost 


greater than $200 million, or otherwise designated by 
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SECLER, Most DCPs cover tne decision the Secretary of 
Defense is being asked to make, the alternatives, the 
progran justification, the program Management plan, 
projected costs, risks, and the limits of the grant of 
authority from SECDEF (thresholds). The DCP is limited to 20 
pages and is essentially a contract between SECDEF and the 
Navy. Programs designated by the Deputy Director of research 
& Engineering or other appropriate principal of DSARC are 
Similarly documented by Program Memorandum (PM). Each is 
developed through interaction of DDR&E, СНО, CNM and the 
Principal Developing Activity (PDA), which is a field 


installation. 


The life cycle of a weapons system, as defined by 
DODDIR 5000.1 (Ref. 2) consists of 5 phases. They аге: 
Conception, Validation, Full Scale Development, Production, 


and Deployment. 


The Conception Phase, conducted at the discretion of 
the Department of the Navy, establishes a technological, 
military, and economic base for the program through analysis 
and experimentation. It culminates with the approval of an 


Advanced Systems Concept. 


The Validation Phase, often identified with advanced 
development, consists of hardware development and extensive 
analysis of the concepts of Phase I. Ideally, it ends with 


a successful demonstration of a "brass board" model. 


In the Full Scale Development Phase the weapon system 
and all support items are designed, fabricated, and tested. 
It includes T&E both on the part of the contractor and the 
Navy. 


During the Production Phase, the weapon and all support 


items and spares are produced for operational use. 
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The final phase, Deployment, consists of providing the 
weapon System to operational units and its subsequent use by 
them. 


In view of the above life cycle, the DCPs are aimed at 
major milestone decisions between phases. The first 
decision, DSARC I, comes after the Conceptualization Phase 
and is essentially a program initiation decision. DSARC IIA, 
initiation of Full Scale Development, confirms that the need 
for the system is current and that development risks have 
been identified and provided with solutions. The production 
decision may be either for a limited (pilot) production 
(DSARC IIB) or full scale production (DSARC III). Тһе major 
elements of this decision are: the need is still current, a 
practical engineering design has been achieved, technical 
uncertainties are resolved, T6E has determined operational 


Suitability, and the remainder of the program is realistic. 


DSARC membership includes DDREE, the Assistant 
Secretaries of Defense for Installations and Logistics and 
Comptroller, and the Director of Defense Program Analysis 
and Evaluation (DDPA£E). The Deputy Secretary of Defense and 


other ASDs may be included as required. 


The DCP eventually forms the basis for the Project 
Manager's overall supervision of a specific project. Тһе 
Test and Evaluation Master Plan must be carefully 
constructed to ensure that all key issues in the DCP are 
addressed and all questions are answered. Thus the DCP 
beccmes an important Gocument in determining where and how 


the T&E for the project will be conducted. 


NAVMATINST 3960.6 (Ref. 11) states that the TEMP should 
be considered as a planning document rather than a control 
document. It contains objectives, issues, responsibilities, 


interfaces, and schedules. It attempts to iđûentify test 
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activities and facilities intended to be used and tasks the 
PM with ensurina all necessary liaison and coordination with 
the test ranges and activities 1s accomplished to facilitate 
advance planning. Тһе instruction requires the TEMP to be 
distributed to the CNO Program Coordinator (or OPNAV 
sponsor), OP-983, COMOPTEVFOR, CNM(MAT 03L,030,033, 04H,04T), 
and participating Systems Commands and field activities. The 


provisions of this instruction apply to all major projects. 


NAVAIRINST 3960.2 (Ref. 12) places the requirement for 
a TEMP on  Less-Than-Major  (LTM) Projects (less than $50 
million RDT&E, less than $200 million production costs) and 
Lower~Range-Less-Than-Major  (LRLTM) Projects (less than $5 
million RDT&E, less than $20 million production costs). A 


Simplified format is provided for LRLTM Projects. 


Hithin NAVAIR, NAVAIRINST 5000.4A (Ref. 13) established 
an integrated NAVAIR Naval Aviation Plan (NNAP) for current, 
mid-range, and long-range planning within estimated fiscal 
constraints. The NNAP is designed to identify worthwhile new 
ideas early through a forecast of the “state-of-the-art іп 
erıtical sciences ПС: уса Топ of operational 
requirements. Specifically, it integrates aircraft, weapons 
and carrier planning data ‘into a long range flan for 
aircraft and weapon development, procurement, and support. 
Additionally, it identifies critical advanced development, 
and provides the Navy wath tong range logistic and facility 
requirements. The inherent security classification of the 
NNAP renders it relatively ineffectual for use at the 


facility. 


The Project Master Plan (PMP), developed by the Project 
Manager/Acquisition Manager, depicts the integration of all 
aspects of weapon system acquisition such as ILS (integrated 


ОСС БИСИ О рро е Rob, DISE, OTSE, procurement and funding. 
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An AIRTASK is utilized within NAVAIR for assigning work 
to field activities. NAVAIRINST 3900.8 (Ref. 14) states that 
an AIRTASK is the document for translating the budget task 
into work assignments to be performed by a field activity. 
When feasible, it includes all necessary technical details 
to enable field activities to accomplish the work in 
accordance with NAVAZR requirements. A WORK UNIT ASSIGNMENT 
is a detailed assignment to a field activity for perforaance 
of a specific task within the scope of a previously assigned 
AIRTASK. 


At the CSD level, in addition to the DCP/DSARC process, 
other types of R&D coordinating papers exist. They are: the 
Techaology Coordinating Papers (TCP) which. highlights 
Service gaps, overlaps and needs in a given technological 
area, Area Coordinating Papers (ACP) which are similar to 
TCPs only cover warfare specialty areas, and Mission 
Coordinating Papers (MCP) which are an expansion on ACPs and 
analyze threat, needed technology, force implications, 


resource impact, and future options. 


C. TEST AND EVALUATION 


The major objective of TEE is to provide the best 
information available on the military utility of a 
prospective system, its operational effectiveness and 
Operational suitability, needs for further development or 
modification, aud data useful in estimating the probable 


cost of development conpletion, acquisition and ownership. 


Basic policy, as ‘stated in Reference 1, dictates 
evaluation by an independent organization so far as possible 
and evaluation of operational Suitability prior to 
Production- Both ОЁ these policies are primarily 


accomplished through the conductance of EOT&E by OPTEVFOR 
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and the conductance of acceptance trials by the Board of 
Inspection and Survey (BIS). These two groups report to the 
CNO for these functions. OPNAVINST 3960.8 (Ref. 5) 
recognizes the Similarity of these 2 functions and calls for 


close coordination to prevent unnecessary duplication. 


The mission of the Board of inspection and Survey is, 


as stated in OPNAVINST 5420.70: 


"uo to inspect new model aircraft for 
construction and suitability for the purpose 
intended and to make recommendations on their 


acceptance by the Navy ". 


The Sub-Board of Inspection and Survey at Patuxent 
River is assigned the responsibility of conducting trials of 
aircraft to determine if the aircraft has been constructed 
in accordance with contract specifications. All materiel, 
performance and design defects and deficiencies found are 


reported by the Board along with recommended design changes. 


TSE is integrated throughout the acquisition cycle and 
is not merely a follow-up to RED. OSD classifies RDT&E into 
two categories: Development Test and Evaluation and 


Operational Test and Evaluation. 
DODDIR 5000.3 defines DTSE as: 


"„.„.test and evaluation conducted to:demonstrate 
that the engineering design and development 
process 1S complete; demonstrate that the design 
risks have been minimized; demonstrate that the 
system will meet specifications; and estimate the 


и мау Чоу when introduced," 


DT&E includes Engineering Development and Design 
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Support Tests, Developing Agency (DA) Tests performed by 
OPTEVFOR, Contractor Demonstration, and Technical Evaluation 
(TECHEVAL). Included within the TECHEVAL is the Navy 
Preliminary Evaluation (NPE), an early assessment of mission 
potential prior to extensive tests, and the Navy Technical 
Evaluation (NTE), a test designed to determine a system's 
acceptability for fleet use. A TECHEVAL, as stated in 
OPNAVINST 3930.8B, is assigned by CNO in response to a 
favorable program full-scale development decision by the 
SECDEF Or conparable CNO/CNM decision approving the 
commitment of resources for full-scale development. 
Additionally, a  TECHEVÀL may be requested by a Developing 
Activity to determine suitability for other acquisition 
programs and improvements such as conversions) and 


modernizations. 


NAVMATINST 3960.6 (Ref. 11) states that OPTEVFOR 
participation in DTSZ is necessary to insure that meaningful 
and realistic tests from the fleet operational viewpoint are 
included in order to permit an early assessment of 


Operational suitability. 
OTSE is defined as: 


"„.„.that test and evaluation conducted to estimate 


the prospective system's military utu туу 
operational effectiveness, and operational 
Suitability (including compatibility, 


Interoperability, reliability, maintainability and 
logistic and training reguirements), and need for 
Devote redt Tons. TIn addition, OTSE provides 
information on organization, personnel 


requirements, doctrine, and tactics." 


An important subcategory of OTSE is Initial Operational 


Test and Evaluation (IOT&5) which according to DOD directive 
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must be accomplished prior to the first major production 
decision. ТОТ&Е is conducted through an Operational 
Evaluation (OPEVAL), which is assigned by CNO at the time 
the TECHEVAL is assigned. The objectives of OPEVAL are to 
determine that the system meets program objectives under 
operational conditions, can be effectively operated and 
maintained by fleet personnel, logistic support is feasible, 
and all performance and suitability questions are adequately 


examined. 


OPNAVINST 3930 В (Ref. 10) defines the various 
priorities which will be assigned to projects ру СКО They 
are: AA-critical to the Navy's readiness for war, A-projects 
of an urgent nature bearing significantly on the Navy's 
readiness for war, and B-projects that are to be started and 


completed as soon as higher priority operations permit. 


31 





III. POINT HUGU'S KOLE I 


|24 


PEST AND EVALUATION 


[tJ 


This chapter follows the organizational development of 
а ТЕРЕ activity, namely, the Pacific Missile Test Center, 
Point Mugu. Many of the organizational problems experienced 
at the Point Mugu complex are similar to those at other T6E 
activities, since they are all operating under the same 
environmental influences. A brief historical background of 
Point Mugu’s past activities is presented. This is followed 
by a more detailed description of events that occurred 
during the past few years. Some of the problems affecting 
Point Mugu's operational effectiveness are discussed 
together with a description of the new reorganization, 


recently instituted at Point Mugu. 


This chapter provides an appreciation fo tne 
difficulties encountered in managing the operations of a TSE 
installation and, hopefully, will provide a better 
perspective of the TSE evolution from the activities! 
viewpoint. It must be emphasized that many OF the 
criticisms and problems mentioned are Strictly the opinions 
of the authors. These opinions were derived from personal 
interviews conducted during trips made to the Point Mugu 


complex. 


A. BACKGROUND 


The Navy's post World War II interest іп ТОЛЕ ӨЛІНІ 
began in 1945 when a group of Navy engineers and technicians 
WaS located there to establish a Navy weapons development 
Gnas test Capability. Shortly thereafter, the Naval Missile 
Test Center (NAMTC) was established to carry out the Navy's 
Weapons test requirements. In 1958, with growing emphasis 
On space programs, the range organizations of  NAMTC were 


reorganized into the Pacific Missile Range and designated 
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one of the six National Ranges under the administration of 
Deputy Director, Test and Evaluation, DoD, with the Navy 
designated as its Range Management Agency (RMA), responsible 
for management and operation. Under the National Range 
concept, ail DoD and other federal agencies obtained common 
range support services free, with the exceptions of services 
performed outside the normal working day or services that 
reguired extraordinary data reproduction. (Ref. 15). The 
remainder of the NAMTC organization was consolidated into 
the Naval Missile Center, as a separate field activity of 
the then Bureau of Aeronautics. Both NMC and PMR were 
separate activities hosted aboard the Naval Air Station; Pt. 
Mugu, which was a suborđinate command of the PMR. NMC 
Maintained its role in weapons test functions utilizing the 
range services of the P4R which played an important role in 


Satellite and space vehicle programs of the 60's. 


As funding for the space program peaked out and with 
growing public concern over the the Vietnam cont Pict, 
emphasis at the PMR again shifted to weapons. NMC became the 
primary user of range services at PMR as test and evaluation 
took a significant role in weapons deveiopment. їп 1970, 
ММС was designated as a subordinate command under the 
Commander of PMR (Ref. 16). PMR's command authority was only 
perfuncory as NMC’ struggled to maintain its identity as a 
Separate command. In June 1974, NNC was again established 
as an echelon 4 activity (subordinate to systems Commands), 
reporting directiy to NAVAIRSYSCOM and hence by passing 
COHPMR іп the chain of command.  NMC's separate command 
identity was shortlived, however, as  COMPMR Successfully 
convinced NAVAIRSYSCOM that the Pt. паа complex would je 
nore effective if the resources of NEC and PMR were 
reorganized into a separate command activity, the Pacific 
Missilie Test Center (Ритс). 
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В. NMC*'S INVOLVEMENT 


In 1958, with the establishment of PMR as a National 
Range, NMC began to expand its capabilities as a weapons 
test facility which entailed functions already available at 
PMR. By 1970, NMC was capable of providing total weapons 
testing services with the exception of the range resources 
needed for actual launches (i.e. launch pads and range 
terrain). The services at NMC included the planning and 
conducting of engineering experiments for determining the 
Operational effectiveness = reliability of assigned 
weapons systems as well as the laboratory investigations 
concerning the development, test, and evaluation of weapon 
Systems. Othér functions, however, not normally performed by 
a weapons testing faciiity had aiso been deveioped. The 
Aircraft Maintenance Department accomplished intermediate 


level maintenance on the growiny fleet of aircraft assigned 


to NMC. The Photo/Graphics Department .had developed 
photographic facilities which constituted one of the 
best-equipped laboratories in the Navy. The Threat 


Simulation Department had developed expertise in the design, 
development, procurement and operation of aerial, seaborne, 


land and special targets. 


C. CONSOLIDATION 


By 1970 PMR's space program invoivenent was all but 
eliminated by the increasing emphasis on weapons testing. 
For purposes of reducing duplicative functions of PMR and 
NMC both activities were placed under one commander along 
with the Naval Air Station and the PMR Facility, Hawaiian 
Area (Figure U). NMC would provide photographic апа 
aircraft maintenance services for tenant activities and 
provide target development and operation support ror the PMR 


Darectorate, 
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PMR, aS a prime supporter of the space program had 
found little difficulty obtaining funding for range 
expansion and improvement. PMR's institutional funding 
flowed directly from the DoD level and resource allocation 
was loosely controlled. AS a member of the National Ranges, 
PMR's facilities were available "free-of-charge!" and, 
consequently, little motivation was provided to improve 
efficiency. Many observers at the Pt. Mugu complex have 
described PMR aS a stagnant organization. One individual 
who was attached to the range organization of NAMTC back in 
the 50's and had recently rejoined the organization stated " 
The range is still conducting business the way it did 20 
years ago." NMC, on the other hand, was receiving much of 
its funding through Navy weapon systems appropriation 
categories. Tighter control of resources through the Navy 
chaia-of-command had a positive effect on NMC's efficiency 
since NMC enjoyed a favorable reputation as a well-run, 
dynanic organization. Command assignment to NMC was’ highly 
desisable and almost an assurance for promotion. Assignment 
to command of PMR waS not viewed in the same light as, 
traditionally, the commander of PMR was on his "twilight 


tour". 


With the exception of eliminating a number of 
duplicative functions the consolidation did little more than 
to bring the Commanding Bingen оГ ОИС funder the 
Chain-of-command of the Admiral stationed at the Point Mugu 


conplex. 


1. Environmental Influences 


анне = сср Tr m ne = о = с саар a p х= Ж ee A ee o > — ы 


Almost concurrently with the 1970 Point Mugu 
consblidation, NMC was placed under the Navy Institutional 
Funding (NIF) system. NNC, as a NIF activity, would require 
customers to pay for the total cost of services provided, 


direct cost plus а share of the overhead. PMR, however, 
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would remain solely imstátutionadldy funded and 
"frez-of-charge" services would continue for range users. 
NMC recognized the new funding technique as a means of 
expaading and improving existing resources. NMC aggressively 
sougat increased business and actively competed with other 
TEE facilities for new weapons program selections. During 
the period of 1970 to 1974 NMC increased its annual business 
activity from approximately $40 million to over $100 
pillion. PHR, on the other hand, had little control over 
improving its existing assets. With defense appropriations 
shrinking, PMR's institutional funding leveled off at $61 
million during the same period, as reported in its FY 74 
budget forecast. Stagnation continued to be a way of 


business at PMR, with no motivation for improved efficiency. 


Since the consolidation, ^ unrest and turmoil have 
hampered operations aboard the Pt. Mugu complex. Lack of 
direction from higher authority was recognized as a primary 
source of discontent. Coordination of TSE activities was 


nonexistent. As a result, T&E field activities functioned 


autoionously, lacking clear objectives on where to 
concentrate their efforts. Fearing exclusion fron new 
programs, SE activities attempted to keep their technology 


base broad enough to permit inclusion in future programs. 
While NEC, under its NI? funding, was able to take advantage 
of the competitive environment and market its services to 
poteatial uSers, the lack of TSE coordination was 
detrimental to PSR. PMR*s inclusion in a program had to be 
detezmined early enough in a weapon svstems' development to 
provide sufficient lead time for development and 
installation of the sophisticated equipment needed for the 
range. Other National Ranges possessed enough functional 
Characteristics Similiar to PMR to threaten its exclusion. 
Since funding for National Ranges was through appropriations 
categories originating at the DoD level, PMR personnel had 


ШЕТІ iience "over their om Ur ©. Frustration 
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permeated throughout the PMR establishment. Personnel felt 
unable to measure their effectiveness in comparison with 
other range activities and, consequently, helpless in 


determining future programs! range selections. 


Although the majority of projects, by dollar value, 
contracted by NMC reguited the use of PMR facilities, there 
existed a definite lack of cooperation between the two 
activities. Communications flow between the two activities 
was оос. Each ccmmand maintained its own contact point for 
external inquiries on the progress of the same progran. 
Duplicative functions again grew. In one instance, both 
commands were developing a data instrumentation package for 
the same purpoSe without each other being aware of it. By 
disassociating itself from PMR, NMC was able to disclaiz 
responsibility for inadequacieS arising during the range 
phases of the TEE. The reason for the rivalous 
relationship, rather than a cooperative atmosphere, wasn't 
сас but perhaps NEC's insistence of maintaining its 


separate command identity was to blame. 


D. RZORGANIZATION PROPOSED 


In 1972, the new Commanding Officer of PMR recognized a 
need to set aside the growing unrest that was affecting the 
entire organization. He cited that lack of direction fron 
above was hindering PR's ability to evaluate itself. No 
clea > dividing line of responsibility between Washington and 
field activities existed. A series of ad hoc committees were 
formed to investigate the difficulties ekisting at the Point 
Hugu complex, while the Command focused its thrust upon 
stirzing up action in Washington. Shortly thereafter, the 
new Commanding Officer announced that he waS proposing a 
reorganization of Pt. Mugu. External support for the 
reorganization was gained by claiming that over 300 jobs 


could be eliminated through the reorganization. Internal 
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pressure against the proposal mounted. Personnel production 
suffered due to the uncertainty of job security. “ne 
conmand billet at NMC, which was. a popular career enhancing 
billet normally held by an Aeronautical Engineering Duty 
Officer (AEDO), was also tnreatened by possible extinction. 
Elimination of the billet was strongiy opposed by the AEDO 


community. 


Before the reorganization obtained enough support for 
implementation, another shock wave infiltrated the complex. 
Early іп 1973, the Shore Establishment Realignment (SER), 
which was successful in closing the Hunters Point Shipyard, 
at the cost of over 4000 jobs, announced plans to turn the 
range functions at PMR over to private contractors, thus 
eliminating 1400 jobs. The employee organization responded 
vigorously to sguelch this latest proposal which vas 
eventually defeated by Congress. Meanwhile, the SER 
incident had dominated the scene through 1973 and, the 
reorjanization never became a reality. Due to the opposition 
from both within and external to the organization there 
existed strong doubts that, even without the SER, the 


reorganization would have been accomplished. 


ERZLATEST SVENIS 


By 1974, it became increasingly obvious to the 
Washington based Navy aierachy that the consolidation of PR 
Mimetz7Oehad accomplished very little towards increasing the 
complex's effectiveness. With the exception of eliminating 
some duplicative functions, the two primary activities at 
Point Mugu had suffered considerable internal strife. The 
arrival ofa new Commanding Officer at Point Mugu, in June 
ОҒ 1974, was overshadowed by two significant events: 1) N4C 
was no longer in the chain-of-command of the COMPMR, but 


instead, was reinstated as an echelon 4 activity, reporting 
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directly to NAVAIRSYSCOM; 2) The National Ranges would now 
charge users all direct costs associated with their 


projects. 


The reestablishment of ММС as an echelon 4 activity 
reasserted the apparent, that NMC was indeed a separate 
field activity. The fact that NÁMC's funding authorization 
flowed directly from NAVAIRSYSCOM had seriously hindered 
COMPMR's apility to exercise any real control over NMC's 
operations. Whether or not NMC's "new found" autonomy would 
have a positive effect on Point Mugu's effectiveness was 


questionable. 


The direct cost to customers scheme would certainly 
have an effect on PHR's mode of operation. Not only would 
range users be able to assess ranges on the less tangible 
Characteristic of service quality but, also, by the more 
objective method of service cost. Whetner or not PMR could 
successfully respond to the new challenge of keeping costs 
down was uncertain, but what was certain is that PMR was 
given a motivation for efficiency and an opportunity to 
evaluate itself relative to other ranges. The stagnancy 
that degraded PMR's effectiveness in the past wasn't 
tolerable under the new runding method, which would instili 
a competitive operating mode among the various ranges for 
the TSE dollar. Not only would cost efficient operations 
mean more business activity for PMR, but also would weigh 
significantly nmuEthe-camount of institutional funding flowing 
from DoD for expansion and improvement of facilities. 
General Starbird, DD(T&E), DoD, has closely monitored the 
Озал ОШ LiStitutiondi funding for the various TER 
activities and has been a strong advocate for efficiency 
measurements of TSE facilities. He has made his opinion 
quate clear; the cost efficient facilities will receive the 
dollars. PMR personnel have recognized the impact of the 


funding change and have initiated efforts to include 
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marketing in their mode of operation. 


No sooner had the events of June 1974 been absorbed 
when the new COMPMR released startling news. He felt the 
present organization wasn't responsive to the Navy's TEE 
needs and that, consequently, a reorganization would һе 
studied. Тһе changes proposed by the Admirai were radical 
and would totally eliminate NMC's identity. TO appease 
opposition from the AEDO community, he proposed that an AZDO 
would hold the second highest billet, vice commander, and he 
would have responsibilities commensurate with the NSC 
command position. Internal resistance never surfaced, as 
with the previous reorganization proposal, since no jobs 
were proposed for elimination. Washington's support was 
achieved by selling the benefits of a reorganized Point Mugu 
complex and avoiding any inferences to inadequacies existing 
at upper echelon operating levels. The new organization was 
to be named the Naval Weapons Test Center (NWTC) thus 
avoiding any inference to the PMR. The reorganization gained 
acceptance. However, just prior to finalization General 
Sitarby rdg a strong opponent of service parochialisn, 
objected to the use of Naval in the title. Navy officials 
Heculesced, and on April 25, 1975, the Pacific Missile Test 


Center (PNTC) was established. 


The 1970-1974 timeframe at Point Mugu waS not a 
productive period in terms of efficiency. Internal 
Managerial problems and lack ог external guidance are 
identified as primary reasons. The PMTC organization is an 
attewpt to solve many of the internal problems and is 


Structured to be more responsive to external activities. 


ВЕСТ ЕС MISSWEE TS5ST"CENTER 


Specific documents addressing the need for 
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reorganization of the Point Mugu complex were not available 
for general dissemination. However, several studies in 
recent |, years have identified the need for improved 
organizational effectiveness of T6E facilities in general 
(Refs. 179 18, апа 19). Тһе data justifying a 
reorganization is limited to personal interviews of 
personnel at Point Mugu and the subjective conclusions 


stemming from these interviews. 
1. Need For Reorganizatjon 


The following is a list of specific problens 
recently identified as affecting the overall effectiveness 
of the Point Mugu complex which, unless otherwise noted, 


refer to the range organization. 
а. Lack of Tecnnical Advice 


The mode of operations on the range has been 
techhically passive. The user requested a certain service 
and received whatever the range was capahle of providing. 
The xechnical competence exists at the range to advise the 
user of what he needs rather than what he wants. Range 
personnel, familiar with its limitations, couid nost 
adequately accomodate a user's requirements by tailoring 


need to match the range's capabilities. 
р. Lack of Cooperation 


Both the range and the missile center have the 
Same basic mission of supporting T&E. As such, both must 
Meee on a cooperative basis for without one there ‘would 
hard.y be a need for the other. However, as the previous 


development has shown, this was not the case. 
с. Unclear External Perception 


То the user who was having TEE performed at 


Point Mugu there was iittle distinction between the ranye 
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and the missile center. When a question arose that needed 
to he answered immediately, he frustratedly discovered that 
he would have to make a distinction between which activity 
to direct his inquiry since both activities maintained 
separate contact points on projects and made little attempt 


to communicate with each other. 
d. Staff Duplication 


This challenges the need for duplicative staffs 
at two activities, located at the same geographical point, 


whose functions are complementary. 
е. Long Range Planning Conflicts 


It is hard to believe that activities at this 
level can justifiably plan for their future involvement in 
particular weapon systems when that determinatioion is 
dependent entirely on decisions made at higher echelons. 
However, lack of guidance from higher authority has forced 
PMR & NMC to plan their future involvements. Unfortunately, 
both activities do this independently, failing to recognize 


their own vested interest. 
f. Personnel Stagnation 


A common complaint іп any organization with 
tenure status is the problem of how to deal with tenured 
personnel who cease to contribute useful benefit to the 
organization. Key managerial positions have been filled by 
the same people for many years who have resisted Changes to 
their mode of operation. One person stated "When a RIF 
(Reduction in Force) occurs, the best people must 40 апа 
we're left with the deadwood. The only thing we can do is 


Баста та еке Organization around then." 
а. Empire Building 


Personnel stagnation has also fostered "empire 


püuilding" with separate units within the organization 
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functioning almost independent of the Command. One 
department promulgated its own five year plan while the 


staff prepared one for the department also. 
h. Improper procedural Implementation 


Changes were generated at the Command level and 
presented to the executive board for dissemination. 
Functional managers did not participate in policy 
implementation. Ideas flowed strictly down the 


chain-of-command without a feedback mechanism. 
i. Inadequate Reports 


The management information system generated data 
inadequate for managers to make decisions. Reports were 
structured in financial terms only, suitable for a financial 
analyst's interpretation. AS a consequence, the comptroller 
was involved in making decisions which should be made by 
functional managers. The comptroller should have only been 
responsible for the operation of set policy and not engaged 


in pianning decisions. 
3. Inappropriate Involvement 


NMC, through its agressiveness, has inherited 
some responsibilities not involved with their primary 
mission. At times, probiens in these areas have tended to 
bog down the Command's effectiveness. The Threat Simulation 
Department, which was responsible for target development 
belonged under the cognizance of the range activity. 
Aircraft maintenance is normally the responsibility of a 


HG ASP Statron and not of a TEE activity. 
2. Corporate Characteristics 
mre = PHTC Organization, recently established at Point 


Mugu, was a radical change from the previous organization. 


Many of the new innovations incorporated were developed 
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following a comprehensive presentation from the Vice 
President of Management Systems at the TRW Corporation. The 
dynanic characteristics of ТРИ, which were considered 
necessary to remain competitive in the rapidly changing 
technological environment, were also considered suitable for 
PMTC's interface with the T&E environment. By instituting a 
TRW-iike organization, PMTC would be acquiring both the 


strengths and weaknesses of TRW'S organization. 


The matrix project mangement method of TRW (Ref. 
20) assigned project managers from the functional 
departments of the organization to manage the company's 
total life involvement in a project. Upon completion, the 
project manager returned to a functional assignment, This 
had proved to be beneficial, in that project managers were 
kept abreast of the latest technology and personnel 
stagnation was avoided in project management positions. On 
the other hand, selection to a project management position 
was temporary and, once the project was completed, there vas 
no assurance that his old position would still be available. 
Consequently, qualified personnel would sometimes shun the 


opportunity for visibiiity as a project manager. 


The matrix project mangement method also provided 
for efficient allocation of personnel. more tlexib11uy 
existed to shuttle personnel to work on other projects when 
not needed on a particular one and, consequently, eliminated 
a large amount of idle manpower. This benefit was traded 
off with the increased cost coliection problem of 
ЦОС уса све costs associated with a particular project. 
Personnel could be working on more than one project during 
the the same period and the problem of cost identification 
increased. One of the reasons for successful project 
completions has been the personal integrity of the personnel 
Which motivated them to work extra hours to get the job 
done. To what degree personnel at PATC will be motivated 


has yet to be determined. 
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The resources of the Naval Missile Center and the 
Paciric Missile Range were combined into the Pacific Missile 


Test Center whose mission is (Ref. 21): 


"To perform development test and evaluation, 
development support, and follow-on engineering, 
logistic, and training support for Naval weapons, 
weapons systems, and related devices; and to 
provide Major Range, technical, and base support 
for Fleet users and other Department of Defense 


and government agencies." 


PMTC is comprised of seven directorates and two 
management groups (Figure 5). Of the seven directorates, 
three are directly involved with the functional aspects of 
current T&E projects and another, the Fleet Weapons 
Engineering Directorate, follows the Life cycle of a weapon 
system after it has completed development at PMTC and is 
introduced to the Fleet. Two other directorates, PMRF 
Hawali, and the Marine Air Retachment, have functions not 
directly affected by the reorganization. A brief discussion 


of tae other directorates and the two groups follows. 
а. Naval Air Station 


An institutionally funded directorate whose main 
mumet Ten of operating and maintaining base facilities and 
provide base support services for the PHTC and ~assigned 
tenant activities will remain relatively intact. In 
addition, NAS assumed responsibility for alıcrart 
Шо гепапсе, M ornerly performed by NMC. The Surface Craft 
Depactment, whose function was to operate, maintain, and 
provide surface craft in support of range operations, range 
Surveillance, and area clearance, was shirted to the Range 


pnaroctorate. 
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b. Range Directorate 


An institutionally funded directorate with a 
Strictly operationally oriented function that provides 
assigned Major Range services, related range facilities, and 
target systems for PMTC projects, including development of 
range systems, subsystens, and techniques. The heart of the 
PMR, the Range Directorate charges users direct costs for 
services rendered. Released from all non-operational 
functions, the directorate assumed responsibility for the 
Surface Craft Department's functions of NAS and the Threat 


Simulation Department's functions of NHC. 
с. Fleet Support Directorate 


Fornerly the Fleet Weapons Engineering 
Department of NMC, this directorate's function remained 
intact. The directorate provides the single entry voint for 
gue Nie in-service engineering projects after completion of 


test and evaluation phases. 
4. Systems Evaluation Directorate 


A NIF unded directorate, the directorate 
provides the technical expertise needed to appraise a 
customer's TSE requirements. Comprising many О tne 
functions of NC's Laboratory Department, the directorate 
performs development test and evaluation, development 
support, test and evaluation Support to in-service 
engineering cf Naval weapons, weapons systems and related 
devices, including development of test and evaluation 


technologies and methodologies. 
е. Engineering Applications Directorate 


А МІР funded directorate, the directorate 
assumed responsibility for the functions of the Data 
Analysis and Processing Department of PMR, some of the 


functions of the Data Analysis and Processing Department of 
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PUR sand some of NMC's Laboratory Department's functions. 
Through its department, the directorate supports the 
prototype design and fabrication needed for PMTC projects as 


well as photographic and instrumentation requirements. 
f. Project Management Group 


Through this group, the project management 
Matrix technique is enacted. The group represents the 
single entry point for PMTC projects providing project 
management, coordination, and information for PMTC projects. 
For adequate customer responsiveness, the group has mirrored 
the structure of the NAVAIRSYSCOM Project Managers! offices, 
a major source of P4TC projects. Similar to the autonomy 
enjoyed by NAVAIR Project Managers, Project Officers work 
directly LOT the Conmander PIC, while obtaining 
administrative support from the PMG. By operating in the 
elassical matrix mode, the Project Officer crosses 
horizontaliy through functional lines in the organization to 
ensure proper project development. The Project Officer is 
not supposed to infiltrate vertically through the 
Organization Бү involving himself in the day-to-day tasks 
which are the responsibility of functional task managers. 
Meeri macy Concerns are time and cost. Project priority is 
set by the Director of the PMG and thus avoids conflicts 
between Project Officers and functional managers over 


ЕЮ Ог ес, traditionaliy a source of conflict. 
Resources Hdanagerent Group 


This group provides a unique aspect to the  PMTC 
organization in that it provides a new approach to resources 
management. The group's function is to provide policy 
guidance, advice, and integrated planning and information 
for the management of the PMTC financial, workforce, and 
physical resources, and provide support to all PNTC 
Management levels. By its method of operation, the group 


encourages participative involvement throughout the 
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organization D acting as a focal point for both policy 
planning/implementation and resource allocation. The RMG 
ШІП best be described as a matrix organization in reverse, 
đistinguishing itself from the normal matrix/staff 
organization. An example would best serve to point out the 


differentiation: 


Under a matrix/staff organization a department 
head, citing a need for a financial analyst in his 
department, would set about to hire a financial 
analyst. After finding someore that appears 
suitable for the position, using his limited 
knowledge of finances aS a criterion, he hires the 
person. The newly hired person works for the 
department and is evaluated based on the department 
head's appraisal of his work. The corptroller, ina 
staff position, has no direct control over the 
analyst's performance and exercises minor control 
through the department head. 

Under PHTC's organization, the head of the RMG'S 
Conptroller/Financial Mangement Department cites a 
need for a financial analyst ina department. Ail 
the financial analysts: work directly for him and, 
conseguently, he is able to stift them wherever he 
deemS appropriate. Quite often, another department 
пау һауе an overailowance and, an analyst сап be 
shifted to the department needing hin, thus 
precluding the hiring of an additional analyst. Тһе 
analyst works direc ely For the comptroller, 


providing financial services to the department. 


The centralizatinsn of resources has great advantages 
in resource allocation. If a directorate cites a need for 
additional office space, the Property Management Division of 
the RMG can objectively evaluate the request and determine 
ОЕ efficient allocation of additional office space, 


considering the overall benefit to PHTC. Through the Work 
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Force Management Division, personnel talent can be most 
effectively distributed throughout the organization. It 
provides a vehicle for moving people between directorates to 
develop competent Project Officers. The dynamics of 
personnel movement, used by the dev son, Gan also be used 


to eliminate the problem of personnel stagnation. 


The centralization of resource allocation may have 
taken some control away from the directorate heads but the 
advantages of centralization appear to outweigh this 
drawback. Looking from another viewpoint, the directorate 
heads no longer need concern themselves over proper resource 
allocation and can direct their energies to the prine 
objectives of their directorate. Within each directorate 
there 1S a resources office which is responsible for the 
financial functions and which also has personnel reporting 
to the RMG on matters of planning, property mangement, 
Management systems, and work force management. The planning 
aspect of the RMG provides an excellent vehicie for two-way 
communications and policy/change implementation. The 
directorates, through the RMG, state the need for a policy 
Change to the Executive Board, comprising the heads of all 
the directorates and the COMPMTC. The Board reviews ‘the 
proposals recommended by the RMG and makes the decision. 
The RMG then disseminates the policy guidance and manages 
its implementation. Through the RNG, the Command hopes to 
attain participative management throughout all levels of the 


organization. 


6. FINDINGS 


Many of the problems afflicting Point Mugu have been 
eliminated by virtue of the reorganization which eliminated 
separate command identities. Still present are the people 
ere кписіуей іп and caused many of the problems that 


plagued the complex for so many years. The new organization 





provides a better means for the Command to react to the 
changing external environment. Intecnally, the organization 
possesses the dynamic capability to adequately assign its 
resources and obtain their most beneficial contribution. 
What remains to be seen is if the personnel will provide the 
cooperation and the flexibility the new organization 
demands. Only through pérsonnel motivation can the problems 
of technical advise and lack of cooperation be eradicated. 
The external perception of PMTC can best be improved by 
customer satisfaction, again, a function of personnel. The 
problem of inadeguate reports is still being addressed. A 
centralized, highly elaborate Management Information System 
(MIS) is proposed that will provide the proper information 
needed at each level of management to put the decisions back 
in the hands of the functional managers. Inappropriate 
involvement has been reduced by attempting to identify the 
most suitable directorate for functions and by eliminating 
those functions which serve no useful purpose in PúTC's 
mission. The RMG provides an adequate. policy implementation 
and feedback nechanism, enhancing the principles of 
participative management. Additionally, tbe ВМС has a 
rather effective method of eliminating persornel stagnation 
by roving personnel out of key positions under the guise of 


personal development. 


Other TEE activities are also recognizing SLT 
organizational inadequacies and are presently  revampinj 
their organizations for more efficient operations.  NATC has 
also just undergone a similar type of reorganization. Yet 
to be resolved are the inadequacies of upper echelon 
д1їгєс*їоп апа coordination. А separate command at 
NAVAIRSYSCOM is being established to bring the TSE effort 
under a unified command. The COMPUTC is nor proposed to 
Ecouble hat" as ATR O-6R in charge of Ranges 5 Targets, 
while CONNATC s to "double hat" as AIR O-6T, in charge of 
E and Evaluation. This will put COMPHTC in charge of 
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some of NATC's resources and vice versa. This might lead to 
some problems but the final details have not been worked 
yet. In any event, the Navy organization is finally 
recognizing its inadequacies in the TSF area and is working 


towards a solution. 
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Іп this chapter the reader is introduced to both the 
Navy Industrial Fund (NIF) and the Resource Management 
Systen (RNS) managerial Concepts. The Navy's RDT&E 
facilities are normally funded and financially managed by 
one of the two methods, either under NIF or by institutional 
dollars under RMS. Ап examination оЕЁ РИТС 15 then done in 
order to acquaint the reader with the financial environment 


existing at a T&E installation. 


PMTC is presently operating under both financial 
methods, NIF and RMS. This dual fiscal mode for PMTC 
resulted from tne recent merger of the Pacific Missile 
Range, operating under institutionally funded RMS, and the 
Navy fissile Center, a NIF activity. Continuation of the 
existing two financial methods is assured until FY-77 when 


PMTC will. become a modified NIF organization. 


A. NAVY INDUSTRIAL FUND 


Navy Industrial Funding 1S an accounting device that 


Eu 


introduces into the nanagement of government industrial an 
commercial type activities an integrated system of 
accounting according to commericak concepts including the 
Mani zation Of а working capital account. Creation in 1949 
of an industrial fund, NIF, within the Navy Department was 
pursuant to the authority granted to the Secretary of 
Derense under the National Security Act Amendment of 1949 


POnewestaplishiment oz working capital funds in the DoD. 


The needs for working capital funds such as NIF 
developed from Congressional investigation into the 
ИШЕСЕ taciiaties of the Military Departments during 


WWII. tudies made by the various Congressional committees 





Bowmted to a definite lack of adequate cost accounting and 
the need for some means of accurate, yet Simple, cost 
determination. By permitting the existence of working 
capital funds Congress constructed a financial management 
МЕО = “making those officals fully responsible for a direct 
accounting for the money they spend, the costing cf each 
job, and the most economical method of accomplishing the 
Hk" (Ref. 22). 


Objectives and benefits derived from a NIF managed 


installation are as follows in Reference 22: 


"1. TO provide a more effective means for cost 
control over operations and a more effective and 
flexible means for financing, budgeting and 
accounting for such operations. 

2. To promote a greater sense of responsibility or 
cost consciousness in customers of the industrial 
fund instaliations, The buying agency must 
actually pay for the goods and services furnished 
by such installations based on the availability of 
its funds. 

ВО ЕЕ dustrial fund activities to more 
effectively discharge their responsibilities by 
Means of separate financing of operations and less 
cunbersome budgeting and accounting methods. 

4. The managers of industrial fund activities 
Should be motivated to operate their activity by 
the most efficient means becaus there will be 


more appropriate indicators of efficiency." 


СЕЗОН Of NIF to an industrial activity such as a 
Shipyard, ӘНШ ӘЕССІІЕ rework facility, or a research 
installation is accomplished by the formalization of a 
Smarter and the establishment of a working capital account. 


The charter represents the approval of the Secretary оғ 
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Defense for the activity to operate under industrial fund 
procedures. Additionally, the Charter describes the 
activities working capital requirement, volume of business, 
Procedures and rates to be used for, cost allocation of 


requested services. 


A significant part of the charter is the providing of 
Ета] working capital to the new КІР activity. It provides 
the cash to cover operating expenses and the liabilities 
which are assumed initiaily, together with provision for che 
required investment in inventories of raw materials and 
supplies. Working capitai net amount, Subtracting such items 
aS accounts payable for goods and services, and annual leave 
of employees, is then available as a revolving fund to 
finance the cost of goods and services produced for the 
users of the activity. Reimbursement of the industrial fund 
is accomplished by billing the appropriation account of the 


using activity for goods or services rendered. 


À buyer-seller or a contractual relationship exists 
between the industrial fund activity and those organizations 
requiring its products or services. Existence ОЕ а 
buye-z-seller relationship commences with the submission of 
an order by the buyer for a specific product or services and 
its acceptance by the selling activity at an agreed upon 
price to be paid by the buyer. Agreed upon price iS at a 
Standard rate established in the charter. The custoner is 
responsible for budgeting and budgetary control for the cost 
e ena- products and Services ordered from a NIF activity. 
Figure 6 illustrates tne cycle of operations under NIP 


шапалсіпда. 


Rate for services and products will vary among 
Bacılitjes under NIF according to the installation*s unique 
Overhead expenses, use of institutional appropriations, ог 


accounting method used to distribute costs. For example, 
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data gathered on five NIF installations indicated a large 
difference in actual cost passed on to the user. NATC 
Patuxent River and NAPTC Trenton have a rate per hour to a 
customer of $9.47 and $9.82, respectively, and a rate per 
meee total cost to the facility „of $27.15 for NATC and 
$27.49 for NAPTC. In each case there exists a difference of 
approximately $18 per hour. For NAF Indianapolis, NWC China 
Lake, and NMC Point Mugu, both the rate per hour total cost 
and rate charge to the customer were identical: $17.58 for 
NAP, $18.74 for NWC and $18.60 for NMC (Ref. 23). 


Dependence of an industrial fund activity on the 
revenue it earns from its customers to replenish the working 
Capital plus the buyer-seller relationship are two 
incentives which contribute to good management. These 
elements influence an industrial activity and its users much 
the same way as custoners and managers in private businoss 
are influenced by competition and the necessity for economic 
procurement and control of spending. A customer is motivated 
to order only that product or service for which he has a 
real need, since it must be paid for out of his annual 
operating allotuent. Tne contractual relationship pressures 
Management of an industrial fund organization to improve 
ese €Stimating and cost control. DoD Directive 7410.4 of 
Septemper 25, 1972 (Ref. 24) offers the reader further 


merOrmation on Industrial Funds. 


b. RESOURCE MANAGEMENT SYSTEM 


Dobbyroct)we 7000.1, ас stated ir Reference 25, 
reflects a change іп views within the Federal Government 
concerning what constitutes qood performance Dy 
administrators. Now performance 15 judged by 
Beet eirectiveness and output. This is done through the 


upssurement of manragement's use of resources during specitic 





periods by comparing actual with budgeted expenditures. 
Older governmental nanagement systens had different 
objectives and were concerned only/with legitimate use of 
public funds. New changes in performance criteria require 


corresponding changes in management and information systems. 


The Navy response to the Directive was its Resource 
Management System. KMS 3s a managerial procedure which 
improves the planning, control and optimal utilization of 
scarce resource (money, material and manpower) at the lowest 


practical work level of an RDT6E activity, identified as a 


| cost center. Participative management is encouraged within 


the RDT&E facility and between the activity and its Systems 
Bomuand Headquarter. Both the cost center and activity 
command management receive greater degree of responsibility 
and a larger roie in decision making concerning what work 
they do, their budgeting request, work schedule and 


progress. 


Several advantages over previous management systems are 
accomplished by RMS. Reporting efforts under RMS show data 
on the same basis as the budget. There is a basis for 
evaluating performance cf personnel, equipment, material and 
PeosectS ап relation to cost. A history or data base for 
future planning is created. Management decisions are made 
more accurate and easier by hard data (Ref. 24). fhe 
improvements are a result of cost center activity in 
planning of resources, execution of the budget, the 
ARCO INN Oor resources, and financial responsibility. RMS 
combines the expanded managerial functions of cost centers 
into a single system in which the functions interact with 


each other. 


For task assignments under institutional funding, MS 
encourages a more objective comparison of task output 


against the resources put into the task. Institutional 





funding, sometimes called "bucket" funding, provides to an 
activity through its "Approved Operating Budget" a source of 
obligational autnority by which to operate. The “Approved 
Operating Budget" is the appropriate Systems Command 
Headquarter's response (NAVAIR for PMTC) to the organization 
forecast of services rendered on task assignments from its 
Sponsor. Figure 7] is a graphical depiction of the 


institutional funding rlow from Congress to a cost center. 


An activity is not limited to performing services only 
for its sponsor. Jt may receive work request from external 
users, such as OPTEYVFOR or a NIF funded organization. Work 
performed on these task assignments are on a reimbursable 
arrangement. Customers are charged the cest or:the services 
or goods produced for them, which include the direct cost 


and possibly the allocated overhead. 


Under RES, all military labor, direct or indirect, 
including duty 15 considered statistical. Тһе label 
statistical" means military labor is not actually included 
in the total costs of a cost center. It is budgeted, 
however, because it is necessary to know for managerial and 
statistical purposes the actual number of military hours and 
doliars used, and how they are used. Statistical "military 
labor is often described as unfunded military labor (Ref. 
25). 


The mission of the responsibility center, cost center, 
апа sub-cost center determines the nature of the workload to 
be accomplished. їп formulating resource planning 
requirements to accomplish anticipated workload, work 165 
classified into meaningful categories of prograts, 
functions, objectives, etc. This entails the iđentification 
 5рвесі11с objectives, programs, fictions, tasks, or 
projects, establishment of appropriate jow order to collect 


and identify costs incurred, identification of the resources 
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required, and development of time Schedules (fiscal 


quarters) for accomplishment of effort. 


At PMTC, the RMS network consists of  sub-cost centers 
(divisions), cost centers (departments), and responsibility 
centers, which generally equate to directorate level 
organization. The network of cost centers facilitates the 
preparation of the budget, the recording of accumulated 
expenditures and, most importantly, the fixing of resource 


responsibility. 


С. FINANCIAL MANAGEMENT AND FUNDING PROCEDURES FOR PHTC 


PMTC continues to use the fiscal  vrocedures and 
financial management which existed prior to the unification 
of PMR & NMC. PMR, while under the RMS managerial concept, 
received its fiscal procedures and funding guidance through 
two mediums; the Major Ranges and Test Facilities Base 
(МЕТРВ), as specified in DODDIR 3200.11, as vell as RMS, as 
promulgated by the Office of the Comptroller, Navy, for 
RDTEZ activities. MRTFB prescribes a uniform funding policy 
that has been implemented at all 26 member activities of the 
MRTF3. The uniform funding policy conceived and executed by 
the Defense Department states that all direct costs of an 
DD icbivity will be paid by the using activity or 
organization, costs which are not reimbursed by the user, 
such as overhead expenses, will be funded by the DoD 
component who is managing the instaliation, i.e., the Navy 
for MR. 


A job order cost accounting system is required of 
participating members by the MRTFB to support DoD uniform 
funding policy. Such a job order system does exist at  PMZC 
POr DON RID and NIF directorates. Once a work request is 


meecoted by PHIC, an Operation number is assi&ned to it. 
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Then all labor and material used on the work request is 


recorded to that specified operation number. The system 
sinplifies the audit trail but necessitates a high degree of 
data recording. 
Directorates falling under ИКТЕВ/ЕМ5 are those 
comprising PMR former functions and personnel plus the 
Program Management Group and the Threat Simulation 
Department from NMC and the Naval Air Station organization, 
including the Aircraft Maintenance Department (AMD). AMD 
was a separate department in NMC and was operated under NIP. 
NIF managed directorates in PMTC number only three, with the 
heart of NMC transfered to the 


Directorate. 


Systems Evaluation 


Following is a list of the directorates and their 
personnel level, Prior to consolidation, PMR work force 
consisted of 556 military personnel and 2422 civilians and 
the  NMC manpower 


complement was 483 military and 1,937 


Civilians (Ref. 26}. 


Directorate officers Enlisted GS zotal 
ПЕТЕВ/ЕН5 

COMPMTC 3 2 105 110 
Flag Admin. 7 2 35 qu 
Liaison Office 0 0 3 
Range/Target Off. 1 0 0 1 
РИС 7 0 70 ШІ 
КМС 1 0 298 299 
Range 38 128 1992 1250 
NAS 48 595 17 2020 
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Directorate Officers Enlisted GS Total 
Marine Av. Det 0 0 2 

PMF (Hawaii) 29 74 58 15 
Total MRTFB 125 801 3040 3966 
МІР 

Sys. Eval. 30 32 704 766 
Ее Npn: ENg- 5 240 245 
Eng. Appl. 2 13 741 138 
Total NIF 37 45 1685 1767 
Total PMTC 162 846 4725 5733 


MRTFB/RÁS directorates receive their operating dollars 
via the Naval Air Systems Command from the Navy RDTGE 
Be ropriated institutional funds as an RNS activity. The 
operating budget is examined and approved Бү higher 
authority within DoD and ' Congress under the policy of 
incremental programmiug of funds. Incremental progranning 
provides that only those funds required for a given fiscal 
year are included in tne authorization request. This 
compels the МЕТРВ directorates to program the funding for 
their RDT&E effort on an annual incremental basis as opposed 
to the fully funded program basis of the procurement 
BppEOprriation (Ref. 2%). Directorates? approved operating 
budget, constituting a separate and distinct program elenent 
ЕПП the DoD budget, is the limit of their obligational 
authority as set ру Congress. ШПРИЦ ораса торат 
authority represents the maximum dollar value of goods апа. 


Services which may be ordered during the fiscal year. 


ШИЕ ИО Па funds make up a large part of the 


operating revenues but not all of it. Approximately a fifth 


64 





of the dollars needed for day to day operatios come from the 
range's users through direct cost charges to them (Ref. 28) 
. This direct charge to users is a recent phenomenon 
resulting from a Defense Defartment study commencing in 
April 1972, chaired by Ur. George W.  Bergguist, Deputy 
Assistant Secretary of Defense (Systems Policy and 
Information), of the funding policy existing at the 26 major 


ranges and test facilities including, at that time, PMR. 


One concluSion reached in Mr. Bergguist's investigation 
was the necessity to improve cost control and econonic 
efficiency among the MRTFB by charging the users the direct 
cost of services or goods ordered. A direct consequence of 
this study was DoD Directive 3200.11, dated June 18, 1974, 
whica spells out that “all DoD Components and other Federal. 
Government agencies will reimburse the major ranges and test 
Facilities for direct cost". All costs not identifiable to 


a user are charged to institutional funds. 


Appendix A projects a clear picture of: the growth of 
direct cost; who the users are; and past, present, and 
forecast operating budgets for PMR. Although PMR ceases to 
exist, the amount of institutional funds allotted to its 
range functions (directorates) will continue as part of 
MTS financial planning. DDT&E is the final approval 
authority within the Defense Department for budget requests 
irom any of the national ranges and test facilities as 
assigned by DODDIR 3200.11. 


Hodernization and isprovement funds for the RMS 
directorates come from several appropriation accounts 
including the RDT&E (N) institutional fund апа Other 
Procurement Navy (OPN) appropriations. The PMR side of PMTC 
represents approximately a $150,000,000 investment in range 
NUStrumentation. An average life expectancy or the major 


range equipment is ten years, With instrumentation 
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obsolescence occuring sooner than that. This would dictate 
an annual investment of about $15,000,000 for necessary 


replacement and update to maintain current state-of-the-art 


rang? instrumentation systems. For the past ten years, 
PMR's budget subuissions for range modernization and 
replacement have been cut in half. To make up for the 


deficiencies in appropriations of needed funds and to 
fulfili the projected reeds of the Navy a $153,980,000 Range 
Instrumentation Procurement plan for the years FY-75 through 
PY-81 was prepared (ref. 28). Appendix A gives a breakdown 
of this plan and Reference 28 reports on the major range 


asset acquistions. 


The three directorates under NIF receive their 
operating revenue ultinately from the user via the progress 
payment method. Progress payments is the existing practice 
used by NIF cost centers for collecting accrued costs 
generated [SO WOLK requests, value of inventory 
identified and reserved for the user's order, and amount of 
work in progress. The customer receives a monthly bill for 
these costs. Reimbursement for services’ or goods ordered 
from the NIF cost center by а user within the Federal 
Government 1S accomplished by cross disbursement without the 
use of checks. Eliminating the needs of using checks is 
done by charging the appropriate fund accounts of the user 
activity and PNTC NIF fund the dollar value indicated in the 
biliing. Collection of costs from external customers to the 


Federal Government is accomplished by checks. 


Internal billing between PHTC's cost centers, both NIF 


rules to ensure 


Pt, 


ЕНІ MRTFB, requires a specific set о 
compliance with DoD's MRTFB and Industrial Fund regulations. 
MRTFB's uniform funding policy permits reimbursement of 
ВЕСЕ cost, only, and NIF allows the collection of total 
BoSts. Direct work performed by a NIF cost center for a RMS 


 Есопег cost center is charged on the basis of direct costs 
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plus overhead. Work for another PHTC NIF cost center is 
Chargeable on the basis of actuai costs, only, without 
overhead application. - A RMS cost center charges only direct 
costs to a NIP cost center (Ref. 26). 

Contrary to established NIF principles there is not a 
general overhead expense applied to the total cost. This 
Slight deviation from NIF procedures is one of the effects 
present in PMTC current fiscal operations that is related to 
its dual mode financial management. Instead, general 
overnead is treated as an indirect overhead, allocated on a 
total population basis and added to a NIF directorate over 


head expense. 


NIF directorates receive their funds for assets 
acquistion or improvement by two means. The dollars are 
either appropriated via a budget request or become available 
from the industrial fund. The choice of which alternative 
method to use for equipment replacement/modernization 
depends on the dollar value involved and/or initial 
utilization of equipment. Fund requests of less than 450,000 
пау be financed from the industrial fund with recovery of 
experditure coming from operating costs charged to users. 
Costs of acquistions or improvements areater than $50,000 
will be financed under appropriated funds. Machinery or 
egulsnent authorized within a customer order for fulfilling 
De work task may be financed froa the industrial fund with 
MeSoOVery Tot costs from the customer. Equipment acquired іп 
this manner will Le disposed of in accordance with the 


Weer's ınstruction (Rei. 24). 


Appendix A gives the major users of NMC and their 


pespective funds spent at the facility. 


O TURE DIRECTION Or PMIC FISCAL MNANAGEMCNT 
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For FY-77 and on PMTC will operate under a modified NIP 
Eoncspt called Reimbursable Institutional Funding (RIF). 
RIF, developed by NATC Patuxent River Fimenciel and 
Operating Cost Center, calls for institutional funding of an 
installation overhead generated by its range functions, with 
the range services direct cost paid for by the user. All 
non-cange cost centers will operate under the normal NIF 
procedure, receiving reimbursement by users for the total 
costs incurred. Implementation of RIF at PMTC will cause a 
thorough examination of all directorates for the 
determination of cost centers which have a range related 
task and those which do not. The opportunity to have a cost 
cent2r overhead expense absorbed by institutional funds vice 
customers! dollars could have a striking effect on the cost 
estimates PMTC will give to customers in the futute. An 
indication of this 1s the example stated earlier of five NIF 
activities and the difference in the rate per hour charged 


to a customer and that which the installation experienced. 


By uSing RIF, PMTC and the Navy will be satisfying two 
conditions. The first is the uniform funding plan advocated 
nr. cBergguist's study group for all 26 major T£E support 
activities and implenented by DODDIR 3200.11. Тһе second 
condition being met is tne increased fiscal control and 
motivation a NIF financially organized activity aives to 


manajement. 


The existing cost estimate procedure used in responding 
to a customer's inguiries is scheduled for improvement. Тһе 
present system calls for a PHTC Project гіісет, шарпи 
receiving a proposed work request, to pass on to the cost 
centers concerned a list of services or goods a user seeks 
to have filled. Boch COst center forwards along its 


proposed cost estimate based on the labor, instrumentation, 


и 


Material, and other items involved. The Project OF Fleer, 


4 
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assisted by a finanical analysist, then totais the bids and 


presents PMTC*s proposal (Ref. 29). 


The current system has several drawbacks. A cost 
center cost estimates for Similar services requested by two 
different customers might not be identical since there 
exists inconsistency in who prepares the proposed costs. 
Likewise, conflict could exists between the Project Officer 
and the cost center manager. A Project Officer is inclined 
to present an optimistic price estimate on labor and 
materiai containing no margin for misfortune or overruns to 
a customer. This is in contrast to a cost center manager's 
compelling motivation to ensure full employment of bis 
personnel and a shop operating at full capacity, The cost 
estinates a shop manager passeS on for the services 
requested could reflect this view (Ref. 30). The presence 
of either individual's subconcious tendency may cause ап 
inaccurate cost proposal to be presented to the Do OD CE d 


user, having all concerned suffer the effects. 


PMTC is working towards comprising a "catalogue" of all 
Poe goods and services that could be ffered by the 
installation and their stated costs. This proposed cost 
determination system vill necessitate extensive research and 
investigation into all PMTC services rendered and goods 
produced (Ref. 29). Initially, a determination of the 
actual costs expended on completing these functions in the 
past must be accomplished. Once actual costs are computed, 
И: ИСЧО 356 done on costs for both validation of 
Cost averaging and to ensure the efficient use of resources 
in accruing the expenses. ІШІ Сг validation analysis 
will be a reoccurring event to ensure cost proposals stay in 
Pies tren changing resource utilization efficiency, labor, 


ane material rates. 
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ТЕ. АННО EVALUATION ASSBSSIENT 


a e oe oo — — ome oe ашамын яғы a со came SS eee — — -—— e e a A—— —D um m == md 


A. MOTIVATION 


Trips made to Point Mugu provided the authors with a 
fairly conprehensive inpression of tne role field activities 
play in the T&E evolution. Personnel at PMTC were interested 
in improving their relationship with Project Managers and, 
consequently, fostering a cooperative atuosphere. Personnel 
intezviewed felt that they were operating in a competitive 
environment and that PMTC would have to actively solicit for 
participation in future weapons programs. To accomplish 
theit objective they felt a need to: first, know what the 
plans for future weapons systems are, second, convey to 
Project Managers what their capabilities are for meeting 
these denands. They did not feel a suitable mechanism 


existed to achieve this objective. 


The existence of a competitive environment has also 
been noted and approved wath restrictions by higher 
authority. The Chief of Naval Material, іп NAVMATINST 
5450.27, stated: 

"...these activities have operated under broad mission 
statenents and often Overlapping functional 
assignments whicn have generated competition for 
resources. While this policy was acceptable during 
a period of transition, continuing progress in the 
establishment and strengthening of “RED centers 
makes it desirable at this time to modify and 
restate policy so as to focus more sharply tne 
ШОГО Sand —errorts of each laboratory and 
смс Coe intens Cf this instzuction to 
Strike a balance between, on the one hand, a rigid 
definition of functions that would create 
monopolies, deprive sponsors of flexibility, and 


aO a decrease in technical options and, on 
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the other hand, a dispersion of capabilities that 
would foster duplication, decrease effective 
application of resources, and promote competition 
for programs and resources. Flexibility and 
dynamic operation are recognized as keys to 
Success in research and development operations. 
Thus the policies enunciated herein are designed 
to encourage creativity, to provide alternative 
technical approaches, to control competition, and 


to ensure coordination of in-house efforts." 


In order to obtain a clearer idea of the relationship 
existing between the user and the facilities, it was decided 
to obtain the opinions of the Project Managers. Road trips 
Бу PMTC personnel to Washington D. С. indicated a wide 
variety of viewpoints existed concerning this relationship 
(Ref. 31). The data obtained was considered insufficient to 
form any definite conclusions. The best method КО 
obtaining additional information would be through personal 
interviews with the Washington based Project Managers. 
Although desirable, this method was discounted due to 
funding and time constraints. The next best alternative was 


to conduct a najl survey. 


Formulation of the questionnaire was predicated оп 
learning the views and opinions of Project Managers in 
certain xey areas of TIGE. Through background investigation, 
research trips, and interviews conducted relative to the Pils 
and their interaction with the facilities, there surfaced 
five managerial functions that needed further exploration. 
Little was understood of the criteria used by PMS іп their 
selection of a T&E facility; what were the factors and their 
order of importance? The latitude a Project Manager has in 
selecting a TEE facility vas not known. It was felt that 
Seteeraints such as technical conditions, project urgency, 


or DCP requirements might dictate which facility a PM must 
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use. The method used by a Project Manager to familiarize 
himself with the different installations was not known. Does 
he rely on past experience, other PMs, publications, liaison 
offices, a staff Within the Systems Command, or a 
combination of these as his information source? Since 
competition does exist between T&E facilities, it vas 
thought important to measure the PM's feelings towards this 
competitive environment. Furthermore, what were their 


opinions on the quality of services provided? 


Once tne categories for PM's responses were deternined, 
formulation of the statements to explore those areas was 
begun. Wording of individual statements was aided through 
discussions held with individuals having T&E experience. 
monstruction and validation of the statements was an 
involved process requiring the selection of response format 
and data analysis to be used, and location of a test 
sampling group to ensure question content and validity. A 
five choice response format was chosen aS a reasonable 
solution to both the response format and data analysis 
requirements. This response format provided ease of data 
reduction and adaptability to the. Statistical Package for 
the Social Sciences (SPSS) on file at the Naval Postgraduate 
School computer center. It also permitted the respondent a 


degree of flexibility in answering the questions. 


After each category was thoroughly covered by its 
statements, an initial version of the questionnaire was 
prepared. Personnei with prior TSE experience were located 
and used as the test Sampling group. Objectives of the 
questionnaire were expiained to the individuals. The group 
was asked not only to complete the questionnaire but to 
communicate their comments concerning the composition and 
validity of each statement to the authors, either on the 
questionnaire or in person. Responses from the sample group 


were used to modify and improve the questionnaire before 
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mailing. Although the questionnaire was principally 
designed and directed to the Project Managers, it was felt 
some of the same issues were applicable to the OPTEVFOR 
community. Hence, the questionnaire was sent to VX-4 and 
МХ 5. 


B. QUESTIONNAIRE 


This section presents the objective, result, and the 
conclusion reached for each question. Responses received 
from the guestionnaire were examined and transfered to data 
cards for tabuiaticn by the SFSS package. Results were then 
printed out in a format similar to that used in the SPSS 
package. Appendix B contains the guestionnaire and input 
data. Abbreviations utilized are as follows: S.A.-strongiy 
IS -agbec, Н.О. по Opinion, D.-disagree, S.D.-strongly 


disagree. 


muescron 1: More than one Test and Evaluation (TSE) facility 
was capable of providing the similar services needed for ву 


росе. 


1. Objective: Desired to establish the existence 
О дәге than опе ТӨ faciliey which had similar 
environment (land, sea, or air), technical support, and 
Divslical plant to conduct the TEE recuired on а 
project. An implied assumption was made that no two 
facilities are identical in capabilities; but when an 
installation's functions are presented in the aggregate 
the summation of those capabilities are approximately 
Sdui@ueGeranoener Installation capability total. This 
information could have been partially retrieved from a 
library research of the DoD's ranges and test 
facilities. А search of that type however, would not 
have indicated the perception held by Project Managers 


and Project Officers towards the existence of more than 
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one T&E facility capable of meeting their needs. One 
of the factors needed to show the existence of a 
competitive environment was duplicate capability at 


various installations. 


2. Results: 


Responses 5.4. A. N.O. D. Sal. 

PM'S 20 19.0% 33.3% 0 28.6% 19.05 

VX'S 20 10.0% 60.0% 5.0% 15.0% 10.0% 

ALL 40 14.6% 46.3% 2.4% 22.0% 14.6% 
MEAN MEDIAN 

PM'S 2.952 2.429 

VX'S 2.550 2.167 

ALL 2.756 2.263 


3. Conclusions: Most managers surveyed agree that 
duplicate capability exists at many installations. 
Most noticeable is the almost even split among the 
Project Managers in Washington towards this statement. 
Since facilities with comparable functions are present, 
it is reasonable to assume the occurrance of 


competition between activities. 


Pest ion 2: The technical constraints of my project limited 


ee -a e 


selection to one T&E facility 


1. Objective: It was believed that the technical 
requirements of a program's TEE could be of such an 
unigue natute as to restrict a Project Manager to a 
Single facility. There are several weapon systems that 
do require special T&E functions and would confine tne 
selection process to only one activity. The relative 
size of the two groups, technically restrictive and non 
restrictive, час unknown. Additionally, the statement 
adas na the individual to judge if duplicate 


ПОСИ е5 exist. 
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results: 


| Responses S.A. A. н.о. D. Sale 
РМ%5 20 15:0% 30.0% 5.0% 40.0% 10.0% 
VERUS 20 10.0% 20.0% 5.05 60.0% 3707 
ALL 40 Es PL 5.0% 50.0% 7.5% 
NEAN MEDIAN - 

DH 5 3.000 92300 
УХ! 5 3.300 Sio SS 
ALL 3.150 342950 

IS Conclusion: Responses from Project Managers 
are the significant ones in this statement. Project 


Managers are concerned with a weapon system R6D and 
developmental T£E in constrast to OPTEVOR*S function of 
operational T&E. RED and developmental TEE contain the 
area oí greater technical constraints. Size of the two 
groups, technically restrictive and non restrictive, as 
viewed by Washington 25 approximately equal. A 
sizeable numbers of projects are compeiled to use the 
опе facility possessing those unique characteristics 
needed for the weapon system T&E. Good correlation 
exists between questions 1 and 2 relative to duplicate 
T&E facilities, i.e., 45% of the Project Managers 
believed they are technically constrained and 48% 
believed duplicate T&F facilities do not exist for 


enrera project. 


Question 3: The urgency of my project limited selection to 


we eee A A deseo ipo > 


Опе тос facility. 


1: Objective: Iwo factors were considered in this 
Statement, time and existing capabilities at TER 
installations. The two elements are directly related 
and dependent on the other. The longer the lead tine a 


project has, the greater is the possibility tnat 





additional facilities could adapt to the T&E 
specification requirements; and conversely for a short 
lead time. As with the objectives for guestion 2, the 
relative size of these two groups, short lead time or 


long, was unknown. 


2. Results: 


НЕА. А. Н.О. De 5. De 

DIS 20 5.0% 30.0% 20.0% 40.0% 5-0% 

VX'S 20 0 5.0% 25.0% 60.0% 10.0% 

ALL 40 2.5% 17.5% 22395 ОО Т. о» 
MEAN MEDIAN 

PATS 3.100 2230) 

үй РС O 32033 

ALL 3.425 3-090 


- 3: Conclusion: Urgency plays a significiant role 
at the Project Manager level and is almost nonexistent 
in OPTEVFOR. Urgency could be another element affecting 
PM's perception of duplicate capability among TEE 


installations. 


Question 4: Current publications provide a useful source of 


m [| | m С ла ыле» 


information on the capabilities of TSE facilities. 


1: Objective: Very littie is understood of the 
technique used ру Project Managers to acguaint 
themselves with the various TSE facilities. There are 
numerous publications, distributed both by the 
installations and the Services, that describe the 
mission, functions, and existing technical assets of 
the T&E facilities. It was felt that these publicaticns 
either do not meet the information needs of the Project 


Mangers or are unfamiliar to then. 


ООШ Есе рс: 
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responses S.A. Ae N.O. D. EOM 


PM'S 21 0 42.9% 23.8% 23.8% 9.5% 
VX'S 20 0 25.0% 10.0% 40.0% 2570% 
ALL 41 0 ЗН ть Т 31.7% 17.1% 
MEAN MEDIAN 

РМ*5 3.006 2.800 
VX'S 3.650 32975 
ALL 3317 3.429 

B Conclusion: Printed information on 


installations and services they can offer needs to be 
improved. Current publications are not providing a 
useful source of information to OPTEVFOR and are 
disregarded by more than half of the PMs. It apvears 
that an improved flow of information to the users of 
TSE facilities at an economical cost is possible 


through better constructed and formulated publications. 


Question 5: Other Project Managers provide a useful source 


of information on the capabilities of T&E facilities. 


1. Objective: In the investigation of the method 
used by Project Managers in gaining a knowledge of an 
IS on Si nice vons and ability, it was suspected 
that other individuais play a roie. A key person might 
be another Project Manager. Possibly his predecessor, 
in the pass down, left either written or verbal 
information as to a ey S capabilities. 
Additionally, existing Project Nanawers, acting as a 
corporate memory, could be a source of knowledge and a 


readily available group to answer questions. 
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Zeeresults: 


ВЕРОНЕ С.А. А. К.О. D. S. D. 

PS 21 4.82 61.9% 14.3% 14.3% 4,8% 

VX*S 20 30.0% 35.5% 10.0% 25:05 0 

ALL 41 17.1% 48.8% 12.027, 19.52 2.4% 
MEAN MEDIAN 

PS 2.524 28231 

УК! 5 2.300 25098] 

ALL 2.415 ze 


3. Conclusion: An important reservoir of knowledge 
and information is an individual's contemporaries. Тһе 
reputation and past associations a facility has among 
this group will. affect future relations. This 
"informal" information link can provide much valuable 
information which would require volumes of technical 
manuals to cover. There is, however, the danger of 
loss of objectivity in the utilization of such a 
source. One especially good or bad incident пау 
greatly Soto and vidas attitude toward an 
installation. There also exists the danger of not 
conpletely weighing all the factors but merely going 


WR tho "current favorites". 


Question 6: Liaisons from T&R facilities provide a useful 


Sobrce of information on the capabilities of their 


facilities. 


Te veost TSE facilities maintain a 
Washington liaison отсе» The office forms a 
communication channel between the Systems Command 
Headguarter and the field installation. It would appear 
that these liaison offices should be a center of 
available information and a Source capable of answering 
ЕС nsger'scuguiries. Ап indication o£ their 


erreectıvchess as a useful source of information should 
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have come from the responses given. 


2. Results: 


| Responses S.A, A, М.0. mio SED 

PITS 21 19.0% 42.9% 9-5% 800% 4.8% 

MEUS 20 20.0% 60.0% 5.0% 15.0% 0 

ALL 41 ЭЕ 51.2% Ub 19.5% 2.4% 
MEAN MEDIAN 

DM S 2/0920 2272 

VS 2.150 2.000 

ALL 2.341 2,095 


3. Conclusion: Liaison between the T&E activity 
and OPTEVFOR appears to be strong and effective. There 
is adequate communication between Washington and the 
ТОБ installations, but not totally effective. A large 
number of Project Managers are not being reached, as 
indicated by the 38% negative response from this group. 
A liaison office which can effectively present current 
information on teeta lation Capabilities. “апа 
technical expertise to the PMs could be a valuable 


asset to a decision maker during his selection process. 


Question 7: A Project Manager's previous assignment provides 
-— c— 0m ee ee me sb eds ero e AO жыға» cee — ee aim: ee i eee ee aim ee deum 


‚а valuable source on information on the capabilities of ?5Z 


faciiiies. 


есте: It vds thought that a Project 
Manager might already have a well conceived impression 
ОЕ existing capabilities at T&E installations. These 
impressions are based upon previous experiences gained 
fron TE related billets or Operational duties. 


Question 36 addresses previous T&E experiences. 
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2. kesults: 


Responses S.à. А. N.O. D. Se D. 

РМ! 5 21. EG) 28-65 98550 23.8% 0 

vX'S 20 35.0% 30.0% 10.0% 10.0% 15.03 

ALL 41 36.6% 29.3% 9.8% 17.1% оо 
NEAN MEDIAN 

РМ15 2.190 1.917 

VX'S 2.400 2.000 

АШ. 2.293 1.958 


3, Conciusion; As stated іп the conclusion of 
Question 5, reputation and past associations between an 
individual and a facility will form the basis for 
Future relations. A significant number of individuals, 
2/3 of the respondents, have formed opinions of various 
TSE inscallations prior to assuming their current 
position. A decision maker's evaluation of an 
instaliation's capabilities and expertise based 
partiaily on past memory contains an inherent  Ganger. 
Past memory can be heavily influenced by one 
Siginificant happening, good or bad, which occured 
during an individual's association with the activity 
and obscure the existing capability level of tne 


facil ty. 


Question 8: Staff assistance within a Systems Command is 


— +’. -. =. — Ae ae ғы © = чс ш 


useful in determining the capabilities of TER facilities. 


1. Objective: Determine if a Project  Hanager 
receives assistance from his Systems Command оп 
тастау selection. it was thought that the Systens 
Command should perform a staff function of aiding a PH 
during his analysis of TSE reguirements and 
installations capable of satisfying them. The Systens 


Command staff should act as an independent centralized 


information source on TEE facilities and, consequently, 
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supplement a PM's small staff with knowledgealbe 


personnel during the selection of a T&E facility. 


2. Results: 


Responses S.A. А. 0 р. S. D. 
Eis 20 5200 553.0% 10.0% 25.0% 5.0% 
үх*5 20 0 50.0% EU 10.0% 5.07 
ALL 40 zu D 52.5% 22427 17.5% 2205 
MEAN | MEDIAN 
Pi > 2.700 2.318 
ЫЕ 2.7700 Е 00 
ALL 2.100 2.405 


3. Conclusion: Some staff ‘assistance is occuring 
at both the OPTEVFOR and Systems Command levels. 
Improved staff assistance might reduce the reliance on 
less than totally objective information presently being 
used by an individual. information received from 
liaison offices, comterporaries, and past assignments 
БЕП uno ЕЛ category of “less than totally 


objective". 


PBuestion 2: There 15 a significant difference in costs to 


таж o ce am p ызы ee — a 


ВЕСОМ Ты fTacilities offering comparable services. 


1. Objective: This statement and the next was 
prepared to help investigate the role played by costs 
in selecting a T&E activity. From research done into 
the fiscal area of TEE, it is known that various 
а ІНЕ сап ofrer Significantly different cost 
proposals. (Тһе unirorm funding policy established for 
PU оее пос: define a uniform cost rate to be 
ЫЧ ОЕШ О orn ТОЛ улпа policy indicates who pays the 
overhead, and leaves the determination of direct labor 
and material rates to the individual ACTIVA) 


Response to the question would indicate the awareness 
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of Project Managers and Project Officers to costs and 
indirectly the importance of costs in their selection. 
Queston 16 produces futher information on ап 
individual's concern for cost proposals. Comparable 
services was used to emphasis that the the variances ir 


costs were for similar work requests. 


2. Results: 


Responses S.A. А. N.O. D. S.D. 
PM'S 21 19-05 3 3995 38.1% 9995 
V'S 20 30.0% 30.0% 23.0% 15.0% 
ALL 41 24.4% 31.7% 31.7% 12.2% 
MEAN MEDIAN 
PM'S 2.381 2.429 
Wt s 22250 Do O7 
ALL 252588937 2.308 
ICONS MUS TON: A majority of those responding 


indicate their acknowledgement of substantial cost 
differences. What is interesting is the large 
percentage of no opinion registered to this statement. 
It is hard to conceive of stimulating a blank comment, 
no opinion, towards the question unless one is 


unconcerned With costs. 


Question 10: Different funding techniques (NIF/RMS,etc) used 


БЕИ КГА езпге tends to mask the actual TSE cost of a 


project. 


1. Objective: As discussed in Chapter 4 on 
financial management, costs paid by a PH could be 
either direct or total depending on what funding 
eeen RIE Or МЕТР, the installation is under. The 
fundgiposnebheg in relation to costs projected to a user 
is immaterial to Services and utlimately DoD, as the 


Defense Department must budget and pay for all costs. 
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ТЕ а Mrcject Wanages was not aware of these 


could Ге 


surface was projecting the 


greatest 


Chapter 4 Section 1 and the exampie 


activities? 


Swayed to 


total 


2. Results: 
Responses’ S.A. 
PM'S 21 38.1% 
VX'S 20 40.0% 
ALL 41 39,0% 
MEAN 
PM'S 1.952 
VX'S 1.950 
ALL 1.951 
Se Corclusion: 


effects certain funding technigues have on masking 


actuai to 


There appears to 


responses 


Significant cost differences, especially 


Managers. 


response on a more soSphiscated statement such 


one, but 
explain 


justify a 


ч= = — ә» ee ee eee Sa = 


project а pro 


selection. 


1 


tal costs 
be 
ins this 


One 


the 


select a 


costos 


Most 


would 


opposite 


facts, he 

that on the 
but the 
referred back to 


the МІР 


facility 


least cost, has 


The reader is 


of five 


rate per hour to customers and total costs. 


А. DS D. SD. 
33.3% 23.8% 4.8% 0 
25.05 353907 0 0 
29.0% 29.3% 2.4% 0 

MEDIAN 

1:857 

1.900 

1.875 

individuals are aware of the 


the 


incurred on a project to the Bon: 


some inconsistency between the 
question and the last one on 
from Project 

expect a higher no opinion 

as this 


happened. It is easier to 


a lack of knowledge in a specified area then to 


ШӘСК ОС concern. 


Once a Tok facility 


ест 


Objective: 


has been chosen for a 


Manager hes no latitude in altering the 


Once a selection was made 


designating a to certain TCE 


ТЕ 


facility perform 


firme ТОП, was not known if this selection could be 
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altered. Futhermore a doubt persisted as to the 
existence of Project Managers and Project Officers! 
authority to execute such altering. It was assumed 


that the change in T&E facilities was due to cause. 


2. Results: 


Responses S.A. À. N.O. b. Ss. D. 
PIES 20 10.02 250% 5.0% 990% 5.0% 
VX'S 20 5.0% 10.0% 40.0% 40.0% 520% 
ALL 40 7. 5% 17.5% 22. 9% 47.5% ES 
MEAN MEDIAN 
PINES 32200 2346932 
VX'S 3:300 33175 
А1,1, 351) 3959253 


3. Conclusion: Selection of a T&E facility саж be 
altered anā a Project Manager 4оев һауе the authority 
to execute the change. Responses from CPTEVFOR present 
a mixed picture as to the latitude of a Project Officer 
in changing T&= facilities. Approximately half believe 
they do possess the authority and the other half does 


not know, as represented in the no opinion percentage. 


Question 12: As regards to facilities offering comparable 
services, there 1s a Significant difference in the 
information feedback provided to a project manager to 


— wir Ge de ne ue mo ee ee oe oe —— wm ie ee ee oe 


effectively monitor the progress of his project. 


1. Objective: There presently éxists varying forns 
of management information systems at T&E installations. 
A management information system should feed useful and 
timely data not orly to those on the station but also 
to a PM, who is ultimately responsible for his project. 
It was not known if wide differences of quality existed 
in management information systems. The existente of a 


better information system which givas a Program Manager 


Вц 





improved control over his program, might persuade him 
ое лесе тала Фок with that facility. This could he 
especially important to a manager from the  wWashinaton 


area working with a distant facility. 


2. Results: 


Responses 5.A. A. Dg D. S.D. 

DNUS 20 25955 30.0% 40.0% S 0 
МХ 5 20 15.0% 25.0% 40.0% 15.0% 5.05 
ALL 0. 20.0% 27.5% 40.0% 10.0% 2.5% 
| MEAN MEDIAN 
Puts 2:250 "2.398 
VITS 2:100 2.50 
ALL 224.15 279903 

CONC ILUSION: The objective was not met due to 
poor formulation of the question. Some believe good 


information systems do exist. Others did not know how 
to interpret the statement, as indicated by the large 


no opinion response. 


Question 13: There is a significant difference in the 


technical expertise of T6E facilities offering comparable 


Services. 


ШОО лес тс: TCE activities âo offer similar 
functions and capabilities, but this does not give а 
Complete picture of their ability to perform T&E. 
Personnel experience and reputation play an important 
n ON MENDES Success tul completion of a program. Do 
differences in technical expertise and reputation exist 


between two duplicate capability installations? 





2. Results: 


Responses S.A. À. NO. D. SaD; 
В 20 30.0% 45.0% 10.0% 15.0% 
VES 20 20.05 20.05 30.0% 30.0% 
ALL 40 25:0% 32:5% 20.0% 22.5% 
MEAN MEDIAN 
PNIS 2.100 1.944 
VX's 2.4100 2:833 
ALL 2.409 22209 
SE Conclusion: At the Washington level a 


perception of different technical expertise strongly 
exists. Іп the minds of the Project Managers, some T&S 
facilities have superior personnel, and to take the 
results a Step futher, superior quality of 
capabilities. OPTEVFOR projects a mixed feeling and no 


other Conclusion can be drawn. 


Question 15: As regards to facilities offering 
comparable services, there is a significant ditference 
in the ease of obtaining non-routine information on the 


progress of a project. 


1. Objective: Apart from the periodic reporting on 
ЕС о status tne Project Manager often has need 
for additional information not contained in these 
Be ports. Personal interviews have indicated sone 
Н.Г CUtalning non-routine information. Іп 
particuliar, some activities have more than one contact 
point ror ZZ ınrormaLr on concerning a project. This 
question was designed to measure the ease of obtaining 
non-routine information. Results could be useful in 
determining the overall coordination and communicative 


feedback existing within the activities. 
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2. Results: 


| Responses S.A. A. N.O. D. SD. 

PAS 20 25.09 25.0% 30.07% 20.0% 0 

vX's — 20 25.03 40.0% 35.0% 0 

ALL 20 25.0% 3215» 32.5% 10.0% 0 
MEAN MEDIAN 

РМ 5 2.85 222 

VX'S 2.1 ааа 5 

ALL 2.215 2209 


3. Conciusions: This ‘question appears to have 
raised a valid point since only ten percent of the 
respondents indicate disagreement. Interestingly 
enough, none of the VX respondents disagreed which may 
гаісе doubt as to facilities' overall cooperation with 
Ge rational TEE units, Perhaps the iower rank of VX 
Project Officers in comparison with Project Managers 
might have an effect on the ease of obtaining 
non-routine information. Results from this question 
indicate that some facilities have done a better job of 
providiing information services to customers than 


otners. 
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Question 15: Funding level for T&E facilities! modernization 


~= лаана Se ee ee ча A et 


is adequate to Support my project's needs. 


1. Objective: Recent cuts in the Defense budget 
have resulted in the curtailment of modernization and 
expansion of TGE facilities. The effects of these 


cutbacks may have already begun to decrease TSE 


с. 


facilities! effectiveness in meeting a user's needs. 
This question was designed to evaluate the degree to 
which existing projects have been affected by a 


reduction in Defense spending. 


2. Resuits: 


Responses S.A. А. NE De В. 

PM'S 19 252% 36.8% 36. 8% 075 10.5% 

VES 20 0 25.05 40.0% 25.0% 10.0% 

ALL 39 O 30:8% 38.5% 17.9% 10.32 
MEAN MEDIAN 

PES 2.842 2.714 

УХ45 Sie 2 3 125 

ALL 5 025 DESEE 

5 Conclusions: Overall, opinions on this question 


were evenly spread with VX respondents indicating боле 
dissatisfaction with project support. Since there is no 
previous data to indicate how Project lianagers/Officers 
felt in the past concerning funding adequacy no vaiiéóc 
conclusions could be nade as to whether cutbacks were 


Significantly affecting program support. 
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Cost Estimates 
Personnel Expertise 
Teenmical Capability 
Logistics 

Program Cooperation 
Test Scheduling 
Others (specify) 


1. Objective: In selecting a T&E facility offering 
comparable services for his project a Project Manager 
presumeably considers the comparability between 
facilities for the identified paraneters. Personal 
interviews indicate that a facility may tend to 
understate their cost estimates in order to attract 
additional business. In any event there may exist a 
wide spread difference between estimated and actual 
costs which would tend to diminish the credibility of 
cost estimates and consequently decrease its importance 
in the facility selection process. The logistics 
Шо еск са пе аш “idencirying to What extent a 
acia сум geocraphical  locaticn and support services 
take in the selection consideration. The other 
parameters are indicatious of the quality of services 
provided which may vary extensively between facilities 
The objective of this question was to determine which 
parameters are most inportant to a Project Manager in 


GCOMetierhingeracility selection. 
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2. Results: 


FREQUENCY OF RANKING (1 = Most Important) 


Selection Parameter Selection Importance 
ШЕ); 8:5 57 
Cost Estimates 1 12 15 7 4u 0 0 
Personnel Expertise SS 3 7 5 0 0 
Tecn. Capability A 2: 21/2 (0 
Logistics 2 1 5 7 915 0 
Program Cooperation ӘСІ 7 6 1 
Test Scheduling ООА сое озо 
Others 00 0 10 0 1 


MEAN 
jM 
2.667 
1.641 
4.667 
ЕТ 
4.486 
9.500 


3. Conclusions: Respondents generally agreed that 


technical capability and personnel expertise were of 


primary importance. Cost estimates narrowly outranked 


program cooperation ¡or the third highest ran 


King: 


Apparently logistics and test scheduling are of little 


importance compared to the other parameters. 


respondents provided additional parameters: 


Two 


one 


"unspecified", the other ranked "ability to simulate 


operational environment" fourth. 


 Езтіеп 17: Rank the following TEE facilities in 


their 


overall effectiveness in providing satisfactory services to 


Project Managers. 
OS ВИЕ сосе = = = 8. Least Effective 


А. Naval Weapon Center, China Lake 

B. Pacific Missile Test Center, Point Mugu 

C. Naval Air Test Center, Patuxent River 

р. Armament Development and Test Center, Eglin AFB 
Е. White Sands Missile Range 

Ee) USS. Army Proving Grounds 

G. Atlantic Fleet Weapons Range 


ECCE Uorcocbastern Test Renge, Patricx AFB 
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1. Objective: This question was designed to 
determine the image different T&E facilities project to 
potential users. The choice was limited to facilities 
capable of providing services to airborne weapon 
systems and hopefully the personnel surveyed would have 
sone familiarity With ог impressions of the 
effectiveness of the respective facilities. This 
question was not designed to determine a facility's 
actual effectiveness but merely to ascertain an idea of 


the reputation enjoyed by these facilities. 
2. Results: 
FREQUENCY OF EFFECTIVENESS RANKING 


Relative Ranking by PM'S 


1 2 3 4 5 6 7 8 MEAN 
NWC 7 2 2 1 1 0 0 0] 237 11 
PMTC 0 7 2 3 1 0 0 0 22216 
МАТС 5 3 1 2 1 0 0 OF 24532 
ADTC 0 0 2 0 2 0 0 О чо 
WSMR 1 1 3 1 0 0 1 07 3:2826 
YUMA 1 0 1 0 9 1 0 1 4.5 
AFWR 0 1 1 0 1 1 1 О А, 
ESTR 1 0 0 1 0 0 1 t 5.0 

Relative Ranking by VX-4 

1 2 3 4 5 6 7 8 MEAN 
NWC 4 1 1 0 0 0 0 DIES 
ОИС 1 1 2 0 0 0 0 О 2.6 
HATE 1 1 2 0 0 0 0 5 222250 
ADTC (Woe. 1 0 0 0 Ü м 
WSTTR 1 1 0 1 0 0 0 1 4.5667 
YUMA 0 0 0 9 0 1 0 O 629 
AFWR 0 0 0 0 0 0 1 Oe 7.0 
BSTR 0 0 0 0 1 0 0 QUELLO 
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Relative Ranking by VX-5 


1 2 3 4 5 6 7 S MEAN 
NWC ФЕ 0: 0 0 0 1.1482 
РМТС EE NEU 222779 79 0 0 2: 
NATC И 1-1 0 0 0 3.375 
ADTC ИФТ 0 0 0 3.250 
ИЗМЕ О 90 о о 0 30 
YUMA ООО ШЕ ООЛУ б 171 © o 7.333 
AFHR БП 20290 © 11 1 5.750 
ESTR DEMOS 00 0 t 2 о 6.667 

Overall Relative Ranking 

1 2 3 y 5 6 T 8 MEAN 
NWC 20. 5 3 1 0 0 0 1 1.7 
PMTC 2 14 5 6 1 A OO SC 
NATC O 2 І 500 оО 
ADTC В оО О 0 32050 
ИЗМЕ E Se 3 X о 1 1 3.538 
YUMA EEUU 7170 5 0 3 257759 
AFUR Ж 1 A с; 
ESTR оО 1 ШС 055626 


3. Conclusions: Overall, "NC, China lake was 
clearly tha most popular facility. Even VX-~4 Which is 
Staeroncduat Point Nagu and primarily uses facilities 
of  PHTC preferred NWC to PMTC. If familiarity caused 
an unfavorable bias towards a particuliar facility this 
did not reduce VX-5'!5 ranking of NAC below number one 
Spot. It is of no suprise that the Navy facilities 
rated highest since respondents were mostly Naval 
Officers. It may be worthwhile for facilities to 
follow-up on these results ana find out what makes 


Other facilities more appealing to potentiai users. 
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Question 18 Increased Competition among T&E facilities 


е 
е 
 — (mi m —— -— 


would increase the effectiveness of services provided to 


project managers. 


1. Objective: In commercial enterprises competition 


has tradıtjonally benefitted customers since it 
inherently fosters nore efficient and effective 
services. In the military, however, an increase in 


duplicative functions may not be feasible due to 
political considerations. This question was designed 
to survey the opinions of users considering both the 
benefits derived from increased competition as well as 
their feelings about such а politically unpopular 


proposal. 


2. Results: 


Responses S-A. A. N.O. D. S.D. 
PM'S 20 10.02 25.0% 15.05 30.0% 20.0% 
УТ 20 тошо 23900 45.0% 45.02 10.0% 
ALL 10 12.5% 25.0% 10.0% 37,5% 15.0% 

MEAN MEDIAN 

DNUS 3: 25 SS 
VG Ss a SO 
AE Зо 250567 

3. Conclusions: In general, respondents did not 


favor increased competition as a viable means of 
increasing effectiveness of services. If a majority 
disagreement had been anticipated it would have been 
worthwhile toas: "Why not №. ІС may be that 
respondents consider outright competition in the 


Mayas an anreasi ble concept. 


Question 19: Competition among TSE facilities should 


uestion 13 be 
AECA оге; 12 reduce the additional cost of 


Overlapping capabilities. 
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Е. Objective: Related to the previous guestion this 
one was designed to solicit opinions For total 
elimination of competition due to cost considerations. 
Hopefully the respondent would weigh the benefits 
derived from competition as it presently exists agdainst 


the increased cost of overlapping capabilities. 


2. Results: 


Responses S.A. А. D D. Se 

PM'S 20 25.0% 35.0% 0 30.0% 10.0% 

VX'S 20 10.0% 45.6% 15.09 30.0% 20.0% 

Л EO 17.5% 40.0% 7.5% 25.0% 10.0% 
MEAN HEDIAN 

PM'S 2.65 2.2144 

VX'S ЕЗ 2.389 

ALL 2.7 2313 


3. Conclusions: Overall, respondents opposed to 
increased competition were in Балар” of =- reduced 
competition. The results may have been affected by 
present public coacern over wasteful military spending. 
It is questionable whether or not elimination of 
competition woulda be financially beneficial in the lone 


run. 


Question 20: Selection of a contractor infiuences the 


—— |— «ады жаға» ee oe чары» 


selection of a T&E facility. 


Objective: Basically self explanatory, this question 
was designed primarily to determine if a contractor's 
geographical accessibility to a facility or preference 
Mc riculrss facility perform the TSE has any 


influence over the actual selection. 
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2. Results: 


Responses S.A. A. О. De Sp 

РН 21 0 14.3% 14.3% 57295 14.3% 

VX'S 20 10,05 20.05 40.05 25.0% 5.0% 

ALL 41 4.9% 17.1% 26.8% 41.5% 9386 
MEAN MEDIAN 

PM'S 3.714 32675 

VX'S Ро 3:0 

ALL 3.341 3.9219 


Conclusions: Although most respondents had no 
Opinion or disagreed it is interesting that at least 
some respondents felt contractors do have an influence. 
It may be worthwhile for facilities to determine ih 
what situations or for which contractors, contractor 
selection has an influence. 


Question 2 DCPS are overly restrictive in their testing 


* 
a 
ee Se Se са- т» A ебаир 


requirements and thus limit the Project Manager's selection 


to cone TSE facility. 


1. Objective: This question had a two-fold 
objective. One was to determine the degree of 
confinement placed on the PM Ly the DCP or if he was 
permitted to utilize various facilities and make an 
effective managerial decision. If the DCP were to 
Соар drca опе ТРЕ facility, other factors would 
become unimportant. Additionally, it was hoped to gain 
some more insight as to the uniqueness of the missions 


or each facility and degree of overlap. It would also 


> 


help to isolate the point of time in the planning cycle 


when a facility seemed destined to get the project. 





2w Results: 


Responses S.A. А. N.O. D. SEDE 

PIES 21 9.5% 303% 33.3% 42.92 4.8% 

VAS 20 0 10.0% 2204 35.0% >. OF 

All 41 4.9% ЭХ 41.5% 39.0% 4.9% 
MEAN ! MEDIAN 

РМ" S 32238 3:429- 

VX'S 34.350 3.300 

All 327293 2353 


Conclusions: Although the question received a 
large number of " no opinion ", there appears to be 
general agreement that most DCPS do not dictate the use 
of one particular facility. The large non-committal 
population may indicate that DCPS are restrictive 
enough to indicate the use of a particular facility but 
nebulous enough that an aggressive PM could  easiiy 
justify the use of other facilities. It appears that in 
regards to satisfying the requirements of a DCP that 
there is considerable overlap in missions of the 


various facilities as interpretted by the user. 


early enough to permit funding justification for facility 


modernization. 


1. Objective: With the advent of the NIF and MRTrB 
funding System, TEE facilities wilt undoubtedly heconxe 
Domos couseadous ІП the selection or internal 
improvements, i.e. only those which will earn then an 
матта С гг: The resulting cavapoility 
uniqueness шау require a greater time period in which 
to prepare for the TEE requirements of a project. It is 
therefore imperative that these requirements be 
identified early in the planning cycie so as not to 


cause delays later on. Although it must be recognized 
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that due to the relative short period of time since the 
Miheroautertonr= = or NIF into ТЕБЕ facilities that this 
uniqueness may just be developing, it was desired to 


get a current basis for the lead-time problen. 


2 НЫЦ 5 


Responses 5.4. A. N.O. D. 5.2. 
POTS 21 0 SOSA 14.3% dos 4 257 
MESS 20 0 15.0% 30.0% 36.0% ZA 
ALL 41 0 26.8% 2 250 34.1% 17.15 
MEAN MEDIAN 
EMES 3. 190 3.3 13 
MOSS 3.650 3.667 
ALL 398515 3:990 


Saeconclusions: Although the PMs appear pretty 
evenly split over this issue, the VX Project Officers 
appear to be currently experiencing some problems in 
this area. TEE facilities may be more in tune to the 
Du EEpects or proerans Than to tbes0T6CE5 requirements 
of OPTEVFOR. Additionally, personal interviews with VX 
Project Officers indicated that scheđule slippage 
during DT&E often caused a compression of time aLlotted 


Tor OTSE. 


Question 23: T&R facilities are consistent in their method 


wind ee сш ee ee Oe oe «ине =<. 


Of assigning project priorities for completion. 


1: Objective: Although OFNAV instructions provide 
general guidance Tor the assignment of project 
priorities, it was desired to ascertain such things as 
if perhaps small projects received less priority than 
arge continulng projects as a result of the financial 
impact on the facility. 1t was also desired to know if 
a project could expect equal treatment at several 


facilities based solely on the merits of the project. 
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2. Results: 


Responses All Most Some Few None 
PM'S 17 0 29.4% 23.256 41.2% 20% 
VOUS 19 0 42.1% ИО 15,8% 10.5% 
ALL 36 0 36.1% 27.8% 27.8% 8.35 
MEAN MEDIAN 
В 3239 39375 
VX'S Zul] 2215) 
ALL 22093 3.000 
3. Conclusions: VX Project Officers appear to 


receive more consistent{good or bad) treatment as 
regards their project priorities. Project Managers tend 
to disagree that there is a consistent method of 
assigning priorities. Perhaps one factor in this is the 
existence of long-term working relationships between 


various PMs and specific facilities. 


' Question 24: TSE facilities show a genuine interest in 


providing program support for successful completion of tae 


project. 


as Objective: This question was designed to 
determine the type of environment a PM finds himself in 
after contracting with a facility. Do they follow 
through with services promised? Are they interesteá in 
his project problems and thus provide the "extras" 
beyond the requested tests? Are facilities more 
interested in acquiring new projects than completing 


the ones they have? 
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2. Results: 


Responses All Most Some Few None 

PM's 19 DIG ZO. 3% ae 0 

VX'S 19 15.8% 47.4% 20415 15.8% 0 

ALL 38 21.1% 4.7% A 10.5% 0 
MEAN MEDIAN 

PMES 2„ 105 2:063 

VOS 223.08 20222 

ALL 2/8234 DOM 

3.. Conclusions: Although the term . "peogralh 


support" пау not һауе been adequately defined, most 
respondents seem pleased with the cooperation they 
жесе тк пешке projects. VX Project Officers may 
not receive the same degree of support in that their 
projects are often grouped together under the 
cognizance of one program manager at a facility due to 


their limited requirements. 


Question 25: Deficiencies noted by the PM in a project's 


did — uw 


progress are examined and corrected by the T&E facilities. 


1. Objective: This was desianed to indicate the 
degree of flexibility of a facility and also its 
мае to ассері constructive criticism by the 
user. It was felt that the user should be assured tnat 
his viewpoint would bea listened to and not dwarfed by 
the magnitude of the TSE business. Additionally, 
facilities shoull have a system by which these 


deficiencies may be reviewed and erradicated. 


DIR EISUTES: 


Responses All Most Some Few None 
PATS 20 10.0% 45.0% 20.0% 20-0 DON 
eS 12 0 99595 27.6% УЕ 0 
ALL 38 d 22252 P /% 29297 220% 
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MEAN MEDIAN 


PS 26050 24399 
VATS 32056 3.100 
ALL 2.842 2202 


3. Conclusions: Тһе results somewhat parallel that 
for program support. VX Project Officers, expectedly, 
do not feel they have the power to cause a change in 
deficient areas at many facilities. Ps adopting a 
more strongly-opinionated(all or none) stand than VX 
Eu ons issue 15 an interesting 


reversal to the normal trend noted in other guestions. 


Question 26: T&E facilities capable of providing services 


a ee a шш ae UMS. mo 


for my project have shown eagerness for my project. 


1. Objective: Eagerness for a project may be shown 
in a number of ways. Two noticeable ways are through 


liaison and scheduling. Reference (31) indicated that 


ОЕША ОСО его of the T&E facilities varied 
considerably. NC was cited as having a very 
well-organized, well-run liaison group in tne 


Washington area. It was desired to ascertain how nan y 
ОҒ the facilities vere able to impress the managers of 
their interests in obtaining projects through  taeic 
liaisons. Bagernsss for involvement шау also he 
detected by the ease іп which PMS can coordinate the 


Schedalıng Of services with a facility. 


2. Results: 


Responses noy 105% Some Pow None 
Puts 20 В SOS 25205 10.0% 0 
М5 ШЕ MOSS 42.1% 26.3% 10.5% 10.5% 
ALL 39 2321% > 25.6% 10.3% 5. 0 
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MEAN MEDIAN 
РИ®5 2. 100 2.000 
VIS 2.684 2.438 
ALL 229185 5220 


Sons mans: Due to the relative difference. in 


dollar value, it is probably not surprising that the 
PMs experienced a greater eagerness from the facilities 
than did the Project Officers of the VX squadrons. What 
that по 


"none", which seens to indicate that facilities are not 


is particularly noteworthy 15 PM answered 


saturated with projects to tne point of not wanting tc 
compete for new projects. 
mest ron 27: TE facilities 


actively participate in the 


selection process of weavon system test assignment. 


ds Objective: This question was designed to 
measure the current extent of " marketing " on the part 
of T&E facilities. Although some of the data on the 


capabilities of the facilities are tabulated in the 


various references cited earlier, it was recognized 

utilization of 
limited 
appeared that additional aid must come from both within 


NAVAIR and the 


that many tests require very unique 


equipment. Since РИ. staffs are in Size it 


Ceen n eS. TEE facilities thenselves 


seom most hignly qualified to tell the managers what 
teney (сап по ror then. 
Za kesults: 
Responses А11 Most Some Few None 
pM 18 16. 75 11.1% 44.4% 11.1% 16.75 
УХ’ > 18 0 27.8% 227555 33.38 11.16 
ALL 36 8,3% 19.4% 30527 22.2% 13.9% 
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MEAN MEDIAN 


DNUS 3.000 3.000 
үх*5 32.78 3.300 
ДЕБ SO S. 0415 
3% Conclusions: Perhaps the tern "actively 


participate" reguired better definition. The results 
seem to indicate, however, that there is not a lot of 
current involvement on the part of the facility. While 
this may be good from the standpoint of obtaining an 
independent decision, it seems that an important source 


of information remains untapped. 


Question 28: What is your rank/GS level? 


de Objective: The objective was fairly 
Straightforward. It was desired to ascertain this 
information in order to have the ability to correlate 
data from any qustion by groups according to rank. This 
also served as a check to see if the questionnaire.had 
been filled out by the decision maker anda thus 
reflected his opinions or delegated to some other 


individual. 


2. Results: 


RESPONSES Оз оп 05 06 07 08 GS9 GS18 GS15 
PLIS 20 1 Zu 1 1 1 1 2 
УХ! 5 29 10 7 2 
ALL 39 10 y 11 1 1 1 2 

3% CONCLUSIONS: Tt appeared that most 


questionnaires were completed by the decision maker cr 
by high level staff assistants and thus it was assumed 
that they accurately reflected the thought process 


behind the selection process. 
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Question 29; Highest level formal education achieved. 


1. Objective: As in the previous question 
additional background information was desired. Couplec 
With question number 30, it provides valuable insight 
on the backgrounds of these very important decision 


makers. 


POSES: 
RESP AA BS/BA US (TECH) MS(NON-TECH) DOC DUAL/MS 


EMS: 20 0 25.0% 50.0% 15.0% 5.02 5.0% 
Ves 19 О 73.7% USES 7o 5.3% 5.3% 0 
ALL 39 O 48.72 jud 10.3% 5.1% ZO 


3. Conclusions: AS might be expected, most PMs 
held a Masters реагее ina technical area and their 
more junior counterparts in VX squadrons held Б5/ВА 
Шет respondents held the equivalent of a 
Baccalaureate or higher. It is noteworthy to observe 
that few respondents held Masters decree irn 
non-technical areas even though a large percentage of 


tneir effort may be in management. 


Question 30: Number of years involved in T&E assignnents. 


ЕЕЕ as in question 28 and 29 additional 


background information was desired. 


2» Results: 


ЕНЕ УУ 3 85 6 9 10 15 25 
Pis 20 4 1 4 4 2 1 2 1 1 
VATS 215, (ЖЕЗ 2 | 
ALL 23 ІІ 2 5.42 1 2 1 1 

MEAN MEDIAN 
PPS I Ups 6.000 
үа 1.882 1.778 
ALL ZO 3.600 
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CO REA os: "Question 7 indicated а hich 
reliance by Pils on their previous experience and 
assrgnaents. The results of this question does not, 
however, indicate an abundance of previous experience 
in T&E assignments. On the contrary, it seems to 
indicate a definite need for a strong liaison between 
the " experts in T&E " and the PM who in many respects 


is acting as a corporate manager. 
БЕРНЕСШІ,Т5 АНр CONCLUSIONS 


Тһе response received from the questionnaire was 
excellent. Preliminary expectations, based on opinions of 
marketing experts, forecast a completed return of 30-505 of 
all questionnaires sent out with possible increases 
obtainable from additionai follow-up. The survey produced, 
however, an overall response of 75%. This tremendous 
response seems to indicate a high degree of interest and 
concern in the areas covered by the questionnaire. It is 
also noteworthy that the questionnaires appeared to have 
been thoughtfully filled out by the decision makers, 


themselves, as indicated by the rank profile. 


Questions 3 and 22 failed to clearly establish the 
degree and type of urgency experienced at the VX level. 
Although the Project Officers did not feel limited to one 
с ас a result of the urgency of their project, most 
agreed that the urgercy prevented adequate time for fecility 


modernization and improvements to complete the TEE. 


ЕО, о, uM 21 attempted to ascertain if and 
why PMNS might be limited to one facility. Over 60% of all 
Er pondemesrelet that more than one facility was able to 
provide the services they required. Of those constrained to 


neta cite), technical constraints was the major cause. At 
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the PM level, urgency contributed substantially to the 
remainder with limitations of an overly-restrictive DCP 


being relatively minor. 


Questions 1, 18, 19, and 26 all addressed the area of 
Competition. Results indicate that a competitive 
environment does exist and installations appear to be eager 
for more business. Competition among facilities as viewed by 
the Project Managers and Project Officers is not favored and 
Should be reduced. Futhermore, it appears the managers 
believe that by nat only reducing but by eliminating 
competition, i.e., do away with installations! overlapping 
capabilities, costs of T&E could be lowered. 

Results of question 17 indicate that Project 
Managers/Officers perceive a difference in the effectiveness 
of various TSE facilities. Results of personal interviews 
by PTC personnel contained in Reference 31 indicate that 
НИС, China Lake, haa a better liaison group and were easier 
ta work with during project assignments. Question 17 
substantiated these opinions Since NWC enjoyed a position of 
number 1 in the eyes of the users. Perhaps the key for 
facilities to improve Project Managers’ perception of their 
effectiveness would be to exert more effort towaris 
formulating IE Сеа но КУ relatioship with Project 


Hhanedgers. 


Responses to the three cost questions, 9, 10, and 16, 
appear to indicate some significant facts. Cost estinates 
are not the number one factor in importance for selecting a 
facility. Most managers are aware of the masking effect 
different funding techniques have on total costs incurred. 
The disturbing feeling caused by a 38% no opinion response 
On question 9 might just be explained by the results given 
ІП questions 10 and 16. Managers in 16 appear to express a 


lack of creditability іп cost estimates proposed by the 
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facilities for T&E projects. This attitude coupled with the 
clouding of true costs by the different accounting 
technigues might generate within managers a feeling of 


AUS Ton mena disinterest towards the fiscal area. 


Question 16 addressed the area of facilities! selection 
criteria. As méntioned, cost considerations did not receive 
the importance anticipated. Technical capability was 
clearly considered the most important parameter in the 
selection process. Question 13 pointed out that 75% of the 
Project Managers felt there existed a significant difference 
in the technical expertise of T&E facilities offering 
comparable services. The next section will address this 
area and propose a method of objectively rating the various 


facilities pased on their technical capabilities. 
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VI. EFFECTIVENESS ANALYSIS 

Results of the questionnaire indicate that for many 
weapon systems: more than one T&E facility was capable of 
Pot riigang the TEE requirements for a particular project. 
Results also indicated that technical capability was tne 
nost important parameter іп selection ОҒ а facility to 
perform a weapon system's T&E. There now exists, within 
DoD, 26 major T&E facilities, many of which are capable of 
fulfilling the T&E requirements of a specific project. То 
the best of the authors! knowledge, no method presently 
exists to objectively appraise the technical suitability of 


these facilities for a particular project, 


This chapter will address the question of which TE 
facility 15 most technically capable of ful£illing the T&£Z 
requirements of a particular project. Solution to the 
question is addressed through use -of a systems analysis 
format (statement of objective, assumptions, and measure of 
effectiveness). Through a systems analysis approach this 
chapter demonstrates an objective method of selecting tae 
appropriate facility for a particular weapon scenario. Іп 
particular, this chapter presents a functional capabilities 
measurement of those Тен racilities capable of providing TEE 


to projects involved in aircraf£t/weapon systems. 


= 
T 
=. 
(U 
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The functional capabilities of the selected faci 
1S quantified based on an appraisal of qualitative da 
The data was based on th2 results of recent trips conducted 
by PMTC personnel to appraise the technical capabilities of 
other ToE facilities. Е ШЕ с emphasized that cosi 
considerations are ignored and that technical capabilities 


are the prime consigerations in the analysis. 


RE OBJECTIVE 
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The objective of this analysis is as follows: 


"Selection of the facility most capable of 
satisfying the TSE requirements of an 
aircraft/missile weapon system іп а capability 


restrained environment." 
We Limitations 


Rarely, if ever, does one facility possess all the 
equipment/technical capability required to meet the needs of 
a specific project. This paper limits its analysis to 
selection of the facility which can perform the majority of 
the services based upon current resources. -In order to 
overcome this limitation a project manager has two major 


alternatives: 


a) Additional funding for facility capability 


expansion. 


b) Selection of secondary facilities capable eue 
providing the specific tests not available at the primary 


вос ету. 


Тһе restriction of the weapon systen to 
aircraft/missile 1s necessitated by the limitations of data 
available. Point Mugu, whose primary mission is tne TSE of 


airborne weapon systems, is the prime scource of data. 


B. ASSUNPTIONS 


1. One ZI£E facility is the "prime contr 


^ 
€ 
= = -— амл C m = р = — > — -— 


ctor’ 
As mentioned previously no one facility has all the 


resources available to meet the total requirements of a 


project, Normally one (load) facility handles the majority 
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of the ТЕ requirements and farms out to "subsidiary" 
facilities specific tasks under the cognizance of the 


Project Manager. 


оп баст can satisfy the TSE 


Although the upper echelon executives of DoD may clain 
that there is little duplication of capabilities within -the 
Department this is not so, It cannot be denied that some 
weapon systems are so unique that the TSE requirements can 
only be met by one facility. It is not these particular 
systems to which this study is directed. The survey 
conducted by the authors indicates that many Project 
Managers feel that more than one facility was capable of 
meeting the T&E requirements of his project. The survey was 
used to make an assessment of the T&E process currently 
employed py the Project Manager. A copy of tne survey is 
included as Appendix B. Results are analyzed in Chapter NS 


Section B. 


B Project Manager's selection not Smee = to 


wes SS = «= чш» — + 32 — Md SU ey EE жыл» pom қағар» «қр» aoe 


moenetuncc3onal ccnstraznts. 


This analysis  presupposes that a Project Manager's 
selection criterion ls based solely Оп РОСТО 
Capabilities. Other overriding considerations may limit a 
Project Manager's selection to a less functionally suitable 
facility. Cost limitations may restict selection to a 
caca сап provide only marginal service. The 
ace may пос be able to accept another project 
due to a backlog of work assignments. Political pressure 
beyond a Project Manager's control may force selection of a 
suboptimal гаса теу иат па pressure From higher 
Ay. Ww Project Manager пау consider additional 


capabilties more important in the selection. 
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It is assumed that the Project Manager is willing to 
use an objective criterion in selecting a facility to 
Ви па Zuture DER requirements. The survey conducted 
indicates a high dependence on familiarity of facilities 
from previous assignments as a selection Criterion: 
Persanal interviews conducted with Project Officers show 
that some Project Officers feit quite adament about using 
the same T&E facility they've used in the past and no matter 
what the capabilities of other facilities are they would 
continue to use the same facility in the future. Here it is 
recognized that preconceived notions have an affect оп 
making an objective decision and that a certain degree of 


bias cannot be avoided. 


5. Project within th 


— ашаң» eye -art a 


(o 


cognizance of NAVMAT 

Analysis will be limited to those projects identified 
Strictly as Navy projects under the purview of tne Navy 
Material Command ( NAVMAT). This avoids analysis of joint 
projects which are concurrently developed by more than one 
service. The authors! unfamiliarity with Air  Force/Arny 


procurement particulars preclude their inclusion. 


6. costs are not considered in selection. 

Although it would be unreasonable to assume that costs 
Should be ignored in the selection ‘process the survey 
indicates that technical capability Significantly outweighs 
considerations of cost estimates in the selection decision. 
Project Managers also agree that the different funding 
techniques used in én funding tend to mask the actual costs 


O Gl Nro ect. 
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Since a cost analySis 1S not feasible at this time it 
must be recognized that the computer program does not 
utilize cost aS a parameter. In particular, funding at TEE 
facilities is in a transitional stage due to the recent 
enactment of the Uniform Funding Policy (1974) and the 


instituting of RIF at these activities. 


7. | Paraochialism should not be a selection criterion 
The feeling that Navy projects should go to Navy 
facilities should not be allowed to creep into an objective 
selection criterion. Although facilities may have priority 
methods that favor their individual Service this 
consideration should be dealt with fter an objective 


decision has been made. 
SDE VELOPUENT OF MODEL 


In order to provide a basis for relating the many 
aspects of capability assessment it is necessary to 
establish some common denominator. А common denominator 
ЕЕЕ ПИ БО 5 ос О Ыы  Lequirement ror this application 1s a 
set of functional capabilities. Investigation into PHTC 
tasks and interviews conducted led to a determination of the 
set of 26 mutually exclusive members as developed in 
Reference 328 Tkis se is intended to exkaust the 
capabilities of a Test & Evaluation activity, including an 
orgranization as large as ВТС, іп terns of 


functions/services. 


The 26 functions are: 


1. Trajectory- Provide precision position measurements 


os aes eS =e nn шз» 


in space, air, surface, and underwater by active or passive 





electromagnetic means, optical means, or acoustical means. 
This includes radar, optics, underwater tracking, missile 
impact location systems, LORAC, and passive tracking 


systems. 


2. Telemetry- Provide for the collection and displav of 
engineering parameters such as onboard inertial data, 
environmental data, scalar miss distance, and other 


per£ormance data by passive electromagnetic means. 


3. Command Control- Provide command control, including 
flight termination control of vehicles and missiles, and the 
remote controi of targets and unmanned vehicles. 

ц. Geophysics- Provides environmental data, including 
geodetic surveys and meteorological and oceancgraphic 


measurements. 


52 Frequency Management- Provide coordination, 
monitoring, and engineering services for electromagnetic 
frequencies within assigned areas. This function includes 
command frequency management, operation frequency support, 


and area frequency coordination. 


6. Photography- Provide photographic, television, and 
technical services required to support the decumentation and 
surveiliance of events occurring at and about the field 
acti ЕЗУ anda the test range as well as the instrumentation 
РИО О ЕС опа dynamic greuwnd and airborne testing 


апа evaluation activities. 
7. Launching- Provide launching of ‘missiles, test 
vehicles, and targets and the ground support including 


ordnance for launched systems. 


8. Recovery~ Provide air, surface, and underwater 
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location and retrieval services. 


9. Operational  Comsunications- Provide necessary 
tele-commprovide capability to conduct and сопігоі, апа 
display data for all operations. unication systems for the 


conduct of operations. 


10. Operational Controls This includes range 
surveillance and clearance, tactical data and real-time 
display, timing, range safety, range control, and range 


scheduling. 


11. Threat Simulations- Provide support to the weapons 
ys tens acquisition programs for developing, testing, and 
evaluation aerial and surface targets simulating realistic 


threat environments. 2 


КЕ ТООП СА Services- Provide prototype fabrication, 
modifications, repairs, calibration, acceptance tests, and 
tenant range services and depot level maintenance © 
existing equipment and provide open shop computer services 


for all authorized external requestors. 


Ew Intermalewechnicalf Servıces- All effort internal to 
the activity, but not directly identifiable with the above 
functional categories. This effort will be prorated to the 


functions: trajectory through threat simulations. 


14. Facilities/Utilities Support- Provide for tne 
maintenance aná repair or roads, buildings, grounds, and 
equionent, and the provision of utility services including 
water, power, telephone, and waste collection and disposal 


systems. 


152 ——Loqristics- Provide for suppiy support services and 


«кек» «кеі» бақыт ml: nen e 


the operation and maintenance of systems which transport 
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material and personnel by land, sea, or air. Provide 
nesessary engineering, tanagement and technical services to 
maintain and support integrated logistic support plans for 


assigned eguiprent. 


16. Air Operations- Provide for the operation and 
maintenance of ali-weather airfields including fire fighting 
and crash crew support, flight support services including 
air traffic control, fueling, other line maintenance and air 


terminal; and search and rescue. 


17. Military Community Services- Provide services such 
as senior officer present/area coordinator, iegal, chaplain, 
dental, recreation, family housing, bachelor enlisted 
quarters, bachelor officer quarters, and clubs for military 


personnel. 


ISE CUELA LE. Provide form the administraticn and 


maintenance of a security program for physical security, law 


enforcement, disaster control, and classified material 
control. | 
19. Non-operational Communications- Provide for the 


administration, operation, and maintenance or long range and 


I xodexeIntsto-pogswmttelecommunications. 


20. Administration and  Banagement- This function 
includes all Command Staff functions involving resource and 
progran management, personnel and financial support 


services, managment systems and data processing 


Еее Assessment- Provide for the acquisition, 


analysis, 


р 


nd assessment of performance and characteristic 
data on aerial Langer ша threat systems and the 
determination of the reguirements, capabilities, and 


applications of aerial targets systems for simulated 
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threats. Support to the aerial target user community with 
technical information concerning the translation of threat 
parameters and weapon system T6E plans to specific target 


requirements. 


22. Software Support- Provide for the management, 


functional, and technical support of all computer programas, 
independent of their storage media, associated with airborne 


кеа роп systems, and related design, Ss аа 


\ 


documentation. 


yes = ee oe oe 


IC on is tens Pest and Evaluation= Provide the 


operational planning, preparation of equipment апа 
гесочгсев, апа performance of engineering ground and 
airbrone testing in static, dynamic, integrated, OE 


Simulated modes to acguire pertinent data to be analyzed for 
evaluation of procedurés, processes, capabilities, and 
Characteristics of weapon systems. The determination of 
design, operational, and compatibility linitations ОЕ 
deviations as they effect the development and production 


decisions in the weapon systems acquisition cycle. 


24. Produc ВОВ ESsuUpport-7 Provide the aggregate of 
engineering assistance requireà by a cognizant field 
activity in support of an ongoing production program 
involving equipments for which it has been assigned 


engineering cognizance. 


Pe eer ange services- Provide basic engineering 
services for design, development, test and evaluation, and 
product quality assessment in support of the developnent, 
production, and operational phases of weapon systems. The 
ЭМСС 21 crudecmemgineerang consideration of compatibility, 
ЕНГ” Сеара у, maintainability, and performance to 


establisn the value of the weapon systems. 
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26. Aircraft Maintenance Department- Provide 
intermediate level maintenance on all aircraft assigned to 
Picmlacilrty, cememactOr bailed aircraft, tenant activities, 
and transient aircraft; and to provide organizational level 
maintenance to the activity aircraft, and transient 


dae crear ts 


There are 26 major T&E/range facilities able to act as 
the lead activity in a weapon system T&E program. These are 
identified in DoD Directive 3200.11 of June 18, 1974. 


1. National Ranges. Those major DoD ranges and test 
facilities which are unique national assets designed to 


support requirements of major DoD programs. 


National Ranges Services 

White Sands Missile Ranges USA 

Kwajalein Missile Range USA 

Pacific Missile Test Center USN 

National Parachute Test Range USN 

Eastern Test Range USAF 

Space and Missile Test Center USAF 

Satellite Control Facility USAF 

Arnold Engineering Development Center USAF 

A a OE Test Facility. Those other major DoD 

test facilities which support, almost entirely, DoD 
requirements. 

fest Facilities Services 

Dugway Proving Ground USA 

Pele tele Ес Me E USA 

Tropic Test Center USA 

| Yuna roving Ground USA 

Jefferson Proving Ground USA 

Electronic Proving Ground USA 





Aberdeen Proving Ground USA 


Atlantic Underwater T&E center USN 
Naval Air Test Center USN 
Naval Air Propulsion Test Center USN 
Naval Air Test Facility USN 
Naval Weapon Center USN 
Atlantic Fleet Weapon Range USN 
Air Force Special Weapons Center USAF 
Tactical Fighter Weapon Center USAP 
Air Force Flight Test Center USAF 
Armament Development Test Center USAF 
Air Defense Weapon Center USAF 


Not all 26 facilities are capable of supporting similar 
weapon systems T&E, but are restrained to one or nore 
differentiable categories such as air related systems, 
ballistic tic siles, ordnance, and elma tic tests. 
Consideration of an activity for ТЕБЕ work is limited іп 
SCO pe to those organizations with related mission 
Statements, technical capabilities, and expertise in the 


broad weapon systems classes. 


One particular segment of activities renders TSE 
support of some degreee towards aircraft/missile related 
systems. It is this general weapon system area and 
corresponding T&E activities that forms the boundary of 
alternatives available for this particular Selection 
mm c M :eNvartrsculasr area of selection was predicated 
Оа аа У озо у зогпа лоп from a TEE activity within 
this segment, PHTC. Of the 26 major T6E/range facilities 13 


dP ene er orn tunctions simuilar to PMTC. 


Table I із а matrix representing the 26 functional 
capabilities and the 14 similarycompeting асбу атс. 
pru ubt 15 indicated by an, x and a lack is 


represented by a blank space. 





Severe data and quantitative measurement limitations 
are evident in attempting to rate a possible 26 functional 
Capaoilities at 14 competing facilities. It was mandatory to 
further reduce the number of similar TSE activities 
available to a Project Manager for selection to the ones 
whose functions have been examined. Information was obtained 
on seven of the fourteen organizations. The seven activities 
аге: РЕТС, Armament Development and Test Center  (ADTC), 
Eglin AFB, Naval Air Test Center (NATC), Patuxent River, 
Atlantic Fleet Weapon Range (AFWR), Roosevelt Roads, Yuma 


Proving Grounds, Naval Weapon Center (NWC), China Lake. 


Determination of a quantitative measuring technique for 
evaluating activities! functions was discovered to ре 
impractical. The amount of technical expertise and TEE 
experience needed to understand the various 26 functions 
coupled with the Lack of a common capability comparison data 
base among facilities precluded the utilization of a 


quantitative measuring method. 


A qualitative rating scheme was substituted. The scheme 
was developed by having individuals (at PHTC) familiar with 
TEE and its respective facilities render a judgement, 
subjective in nature, upon those activities! functional 
ca abilities (Ref. 33). Credance in their evaluation is 
centered upon two known facts. The first element is the 
individual's experience ii TEE and his present work function 
asta "осағгап апайусс. The second iten is the recent 
completion of EET ЕДЕН 56 to the Six identified 


orgranizations, excluding PMTC by a team from PMTC. 
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CAPABILITIES 

Trajectory 

Telemetry 

Command Control 

Geophysics 

Frequency Management 
Photography 

Launching 

Recovery 

Operational Commuaicatiors 
Operational Control 

Threat Simulation 

Technical Services 

Internal Technical Services 
Facilities/Utilities 
Logistics 

Air Operations 

Military Community Services 
Security 

Non-9perational Communications 
Administrative & Management 
Threat Assessment 

Software Support 

Weapons Systems ТЕЕ 
Producton SUpport 
Engineering Services 


AAC a t Maintenance 
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"X" denotes existence of capability at anove facilities. 
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The team was asked to apply a qualitative rating scale 
consisting of four categories; better, average, worse, or, 
no capability, to each ох the functions present at the 
facilities visited. Average was defined as the typical level 
of capability existing for a specified function after 
Süryvyeying all the facilities; due to the subjectivity of the 
investigation, no one activity should necessarily .represent 
the average. Better and worse classifications were easily 
perceived by the team relative to the average. A modified 
ordinal scale was selected as the "yardstick" best suited to 
a subjective qualitative rating technigue. A cardinal scale, 
representative of a quantitative scale with well defined 
intervals, could not be adapted to a judgemental rating 


scheme. 


An arithmetic method was needed for the transformation 
of a qualitative ranking into a numerical score. The ordinal 
Scale easily adapted itself to a numerical scoring process 
and values of 0 to 3 were assign to the 4 classes in this 
eden: ONO capability, 1 worse, 2 average, and 3 better. 
Transformation from a qualitative to numerical scoring 
permits the utilization of a crude measure of effectiveness, 
(ШОР). А MOE is necessary for satisfying the requirements 
of assigning weighting factors to the 26 functional 
capabilities; and, to reach a conclusion as to the overail 


Standings of the alternative activities. 


Utilization of this MOE waS permitted only after 
recognizing three existing limitations: its subjectivity, 
lack of a well defined interval to determine how much better 
is better from average and average from worse, and a rater's 
biasing. Personnel rendering their opinion on the seven 
facilities! functional capabilities are members of PHTC and 
as such may һауе some presuppose tendencies towards favoring 


PHTC. The ordinal value points assigned gives oniy an 
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opinion as to the relative Position of the specific 
functions at one activity to the others. It does not 
indicate the degree of satisfication of a test requirement 


which might be obtained at tne facility. 


Subjective rating scores of the various activities! 
functional capabilities by the visiting team are listed in 
table II. The qualitative judgement scheme used prohibits 
any futher compurational analysis of the scores, excluding 
totalling the points, such as determining a numerical mean 


either by function or overall. 


Actual assignment of value points to the capability 
matrix were by necessity qualitative in nature.  Witnin a 
functional elenent it is felt that consistency exists; i.e., 
relative difference in a functional capability between 
facilities with assigned values of 2 and 3, respectively, is 
approximately the same as petween facilities with assigned 
values of 1 and 2, respectively. However within an activity 
ВЕ dasficubteto justify whether the assignment of a 1 or 
Pene Lunction is analogous to awarding a 1 or 2 to 
woer гппсолоп. біпсе the availability of data constrains 
the research to this £orm of information, an assumption is 
made that a well defined ratio exists between functional 
capabilities! assigned value points. Sensitivity analysis 


may justify or preclude this assumption, 
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Telemetry 


Command Control 
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Geophysics 

Frequency Management 
Photography 
Launching 


Operational Conmunications 
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Operational Control 
Recovery 

Threat Simulation 
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D. COMPUTER PROGRAM 


The computer progran (Appendix C) is designed to 
determine a facility's ability to support a given project 


through a simple linear relationship. 


"The project is initially analyzed to determine the 
degree of importance of each of the 26 functions in the 
completion of its T&£Z. Based on this analysis, weighting 
factors are assigned to each funtional category. These 
factors are then combined with the previously defined 
Capability matrix to form the Linear Summation Equation. For 


examples 


EFAC- FUNC(i) x CFAC (i) 


ШЕКЕ ЕАС 5 the effectiveness index of a facility, FUNC (1) 
is the weighting factor assigned to each functional category 
for the project, апа СРАС (1) 15 the assessed capability of 


the facility in each of the 26 categories. 


Miepeptitaletaecuwity as then the facility with the 
highest effectiveness index. This index can, as previously 
noted, be subjected to a sensitivity analysis and be 
combined qualitetively with other factors such as cost, 


ааз гази су etC: 


The program also has the capability to do a sensitivity 
analysis. Тһе weighting factors for each category may be 
vame ay a Constant ratio, CATI, CNT2, etc. or a constant, 
VARI, VAR2, etc. may be added to each individual weighting 


value. 





ПРОБЕЛ ТӘРАРОПН OSCBENARIO 


The weapon system to be tested 1S an air-to-air short- 
range, dog-fight type nissile With passive homing 
capabilities(infra-red) and its launch vehicle an F-14/VFAYX. 
The missile should be capable of being launched effectively 
within all points of the aircraft combat envelope against 
multiple targets. The missile has a 25 "g" turn capability, 
10 miles maximum range, and Mach 4 speed. Targets should be 
representative of current Soviet combat aircraft (MIG 21, 
MIS МТС 25, SU 9, Blinder). 


1. Specific Requirements 
The specific requirements of the test and 


evaluation are: 


а. Base two fighter aircraft at the test site and 
provide hangar, fuel, oil, ground support equipment, etc., 


needed. 


b. Provide housing, messing, transportation to 


БшЕиОово рота, Crea, and technicians. A total of 30 people. 


C. Provide ordnance storage, handling, missile 


tO 
assenbly and checkont for 20 missiles. 


d: Provide threat Simulation matching the targets 
mentioned above as closely as possible including muitiple 


(2-4) target presentation. 


е. Prmovidemmeoaqmunmication ana command control of 
targets to present realistic maneuvering capabilities while 


guaranteeing the safety of launch vehicle crew. 


Ix Provide six channel telemetry for data 


eOe tOn and missile evaluation. 
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Ga Provide ar ао те photographic package 


capable of documenting missile shots and target maneuvering. 


he Provide limited technical services in the area 


of T&E support for this weapon systen. 
i. Provide data processing for test results. 


> Program expected to be completed six months 


from arrival on base. 


Ze Task Categories 


m Gr Tr тэш» ui ne u | u e Eg 


In order to ensure consistency and establish a basis 
for sensitivity analysis, the 26 functions were grouped Ineo 
4 categories as follows: 

а. MPSS mon SUDPOFt- those | functional 
capabilities required to support a mission airborne weapon 
system T&E: 

{1.)  Aircrazt Haintenance 

(2.) Air Operations 

(3.) Frequency Management 

(4.) Geophysics 

(5.) Operational Communication 
(6.) Operational Control 

(7.) Recovery 

b. Test Requirements- those functional 
capabilities directly related to data collection and test 
conductance; 

(1.) Command Control 
(22) Launching 

(3.) Photography 

(4.) Telenetry 

(Sc Thredt Sinulation 


(6%) Талесвогу 


о 





C: Technicai SU POLt- those functional 


ров № рота the technical "logistics" necessary 
to Satisfy the TEE requirements: 

(1.) Engineering Services 

(2.) Internai Technical Services 

(C. NPIOOGUCGHWOn Support 

(4.) Software Support 

G) тее са Services 

(6.) Threat Assessment 

(7.) Weapon System T&E 

de Base Support- those functional capabilities 

necessary to support routine military operations: 

(1.) Administration and Management 

(2.) Facilities/Utilities 

(3.) Logistics 

(4.) Military Community Services 

(5.) Non-Operational Communication 


СС Ssecuryty 


3: Weighting Factors 


Depending on the weapon Scenario selected, each of 
the functional capabilities has a different degree of 
ШОО ОООО ОШ therefore, has a different weighting in 
determining the overall effectiveness function for each 
lu moyen though Subvective, a icgical and consistent 
approach to assigning weighting factors must be selected. A 
hierachy of importance was initially chosen as shown in 
Table III. The functional capabilities were ordered from 
least important to most important. In this instance both 
Internal Technical Services and Production Support were not 
required and were assigned a weighting of zero. Both 
Frequency Nanagement ang Management and Administration were 
Considered "next in importance. This process was continued 
down the line until the last and most important functional 


capability, Telemetry, was assigned the highest rating in 
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the hierachy. 


This initial approach to assigning weighting 
factors’ was considered too restrictive. Boch. ‚funeticnal 
capability was assigned a discreet weighting factor, 
incrementaily rated one unit more important than the 


рЕссейіпа functional cepabality/capabilities. Depending on 
the decision maker's familiarity with the  reguirements cf 
the scenaric in question, he may feel that more than one 
unit degree of importance should be assigned to some 
capabilities. For this particular scenario it was felt that 
Command Control should be placed two increments over the 
preceding category and that Telemetry should he placed three 
units over Command Controi. The results are presented in 
Table IV. In this particular case, the weighting factors 
now range from 0 to 15. Other personnel, more qualified to 
perform the weighting assignment, might feel comfortable 
with a more expanded scale. An organization such as NAVAIR 
Шо пау ре able то expand the scale significantly and thus 


provide for a more sensitive analysis of the data. 


ANS provide a check on the adequacy of the 
weighting factors, the summation of weighting factors within 
each general category should be compared. For this sconario 


Fur Zresuilts are aS follors: 
Test Requirement 60 
MASSON Support 40 
Bene Support 30 


Base Support 15 


The weighting of each general category seemed to 
Штат е ішдогсалпсе, So, for this particular 


Scenario, ПО Leassessment of the task categories total 
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weighting factor scores was considered neccssary. If it was 
feit that a reassessment мас necessary, an iterative 
approach to reassigning weighting factors would be 
appropriate. A later section will address this approach in 


the sensitivity analysis. 


It must be emphasized that these totals are 
directiy related to the scenario depicted. A weapon system 
which greatly pushes the state-of-the-art might, for 
example, require a much righer value in Technical Support 
whereas a system involving a great number of people might 


require a higher total ir Base Support. 
4. Data Usage 


Once the weighting factors were assigned, they were 
placed into the Linear Summation Equation from which the 
computer program outputted an ordered ranking of facilities 
by technical capabilities. Depending on the degree of 
confidence the decision maker places on his’ weighting 
assignment, he may want to input another set of weighting 
factors to ascertain if they will affect the results. This 
approach is addressed in the next section, the sensitivity 
analysis. If the new set of weighting factors affects the 
effectiveness ordering of the facilities, a reassessment of 
critcal factors is needea. If the order is unaffected, no 
further assessment for determining facilities! technical 


capapilties for the scenario is needed 
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PUNCTTONAL CAPABILITY HIERARCHY 


1. Internal Technical Services 
Prođuction Support 
2. Frequency Management 
Admisistrative and Management 
3. Geophysics 
Logistics 
Non-Operational Communications 
Engineering Services 


4. Facilities/Utilities 


WW мсм NH зе > 


Military Community Services 


Us 


Operational Communications 
5. Security 
6. Recovery 
7. Technical Services 
Threat Assessment 
Software Support 
‘Aircraft Maintenance 
8. Photography 


Launching 


о ЫМ ND D KD до = 


9. Air Operations 


42 


10. Threat Simulation. 
INTE Tee tor y 10 
Operationai Control 10 
Weapon Systems Test £ Evaluation 10 
ПЕС оппаа Солго 11 
13. Telemetry 12 


Table III. Hierarchy of Functional Capabilities 


Barıma от the 26 functional capabilities from least to 


most important in numerical order. 
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1. 
2. 
3. 
4. 
5. 
6. 
p 
B. 
On 
TO. 
11. 
12: 
> 
14. 
Mo 
16. 
17, 
18. 
19, 
20. 
21. 
Dos 
2d 
24. 
23: 
205 


DNUCITONAL CAPABILITY WEIGHTING FACTORS 


Trajectory 10 
Telemetry 15 
Command Control | 12 
Geophysics 2 
Frequency Мападепеп ғ 

Photography 

Launching 

REGOVELY 


Operational Communications 


u 


N wo © X Dm NY EZ WO N о ос о со WMHs A = 


Operational Control 

Threat Simulation 

Technical Services 

Internal Technical Services 
Facílities/Utilizties 
ПОСЕБ 

Air Operations 

Military Community Services 
Security 

Non~Operational Communications 
Administrative and Management 
Threat Assessment 


Software Support 


nad 


weapons Systems Test & Evaluation 
PROdUCE LON oppo] < 
Engineering Services 


Aizserarteigr wenance 


ua M or Functional Capabilities 


Finalized weighting factors used in dog fight missile 


Scenario. 





КТІ ДИЮ SENSITIVITY ANALYSIS 


A computer solution (Appendix C) was obtained for the 
example outlined using the stated values. In addition, 
initial senSitivity analysis was conducted by varying the 
category totals in such a manner as to keep the total point 
value of 145 constant. In all caseS the Base Support 
category was held constant Since it was felt its assigned 
value accurately depicted its importance. Changes were 
uniformly made on all elements within a category by inputing 
a ratio value into CAT? through CAT4. Aiteration of CATT 
through CAT4 investigated the sensitivity £ total task 


categories’! assigned values. 


The first two cases show the effect of first decreasing 
Test Requirements by 16 percent and then increasing it by 
the same amount and evenly distributing the difference to 


Mission Support and Technical Support. 


The next two cases depict the subtraction and then the 
addition of 20 percent (8 points) on Mission Support. 
Constant point value was maintained by evenly distributing 


the difference to Test requirements and Technical Support. 


The final two cases depict a similar subtraction and 


addizsion of 20 percent (6 points) on Technical Services. 


In all cases above the relative ranxing oraer remained 
constant, although the absolute difference en scores 
changed. ADTC closely challenged  PNZC througnout tne 
analysis for the number one position. A maximum dirierence 
in scores between the two facilities remained within three 
percent, The analysis shows that the values placed on 
category totals 15 not highly critical to end results 


thpougnout a reasonable range of values.£$15-20 percent). 
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Weighting factors within a task category were next 
changed independently by inputing an adjustment vector 
YAR): Throughout this analysis the total weighting factor 
of eacn task category was held constant. | This analysis 
assumed a properly assigned category total but checks the 
sensitivity cf assigned functional values within a category. 
In this example only the values with in the Test Requirement 
category were altered but similar analysis could be 


performed on the other three categories. 


First, 5 points were subtracted from the Telemetry 
value and 1 point was added [to each of the other five 
functional areas. It should be noted that this altered the 
functional hierarchy ust hd the category by placing 
Telenetry subordinate to Command Control. Next, the 
category hierarchy was held constant by subtracting 1 point 
from Command Control and 3 points from Telemetry and adding 
1 point to each of the other four areas. Finally, all areas 
Within the category were uniformly assigned a value of 10 


points. 


Results indicate no change in the hierarchy of 
facilities! totals and little relative change between two 
У facilities. Іп this scenario the task category 
total seens to be the dominated factor and little 
sensitivity is shown by manipulating each of the six 
variables Within the Test Requirement category. Sone 
facilities had the same values assigned in their capability 
vector in all effected functional area and thus showed no 


Change in effectiveness total (eg. WSMR). 
appears that Once the faclitiess are accurately 


pa Ооо <capabiiities, a Project Manager may 


assign his own weighting factors with confidence. 
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с. CONCLUSIONS 


Suitable facility selection for adeguate testing of 
weapon systems is an important prerequisite to acquiring 
combat functional weaponry. The present method of selection 
has drawbacks due to tne subjectivity of the selection 
process; parameters somewhat irrelevant to proper selection 
frequently weigh heavily in the final selection. If the 
Navy is to improve its system acquisition process a more 
objective criterion must be implemented in the selection 
process. 

\ 

This analysis has demonstated a rational method of 
objectively assessing the functional Suitability of a 
facility for fulfilling the T&E requirements of a particular 
project. Time constraints and data limitations restricted 
сеи еба ала objectivity Of the analysis. Іп particular, 
the data source was based on the opinions of personnel 
attached to the same facility. This may inherently sway 
their impressions of competing facilities. The restrictive 0 
- 3 rating scale employed eliminated the possibility of 
distinguishing between capabilities that were just above 
menage and those that were totally superior їс their 
competitors. The sensitivity analysis points out the 
importance of an expanded rating scale since weighting 
factor variations as large as 20 percent were ineffective in 
altering the. overail ranking. It is believed that if more 
objectivity and an expanded rating scale were employed a 


differentiation in ranking would have resulted. 


Follow-on study 1S recommended in this area since the 


analysis presents a straight forward method of ranking 


facilities through a weighted linear summation of 26 
en ta itable functional capabilities. To reduce bias, 
personnel at the ПОВЗ ог рор level should conduct a 


survey of TSE Facilities and rank the Pune t10nal 





capabilities оп а scale of 0 - 10. If a facility expands 
its functional capabilities or for some other reason feels а 
higher rating is warranted the facility could appeal for a 
review of its ranking in the questionable areas. This 
ranking method could also provide useful feedback to 
facilities for determining their relative effectiveness and 
help identify areas needing improvement. When a Project 
Manager determines his TEE requirements a few computer cards 
could provide him with a list of the most qualified 


facilities. 


This analysis does not pretend to propose the ultiuate 
solution to the decision maker. Prior to final selection 
the Project Manager might have to weigh other less tangible 
considerations or parameters not yet identified by the 
analysis such as costs, time, and scheduling problems. 
However, this type of analysis reduces the difficulty of 


making the proper selection. 
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VII. CONCLUSIONS AND RECOMMENDATIONS 
This chapter states what was accomplished by the 
research in view of the five objectives covered in the 
introduction. An attemot is also made, where applicable, to 
recommend improvements and to point out areas where 


additional research could be acconplished. 


А. WEAPON SYSTEM ACQUISITION PROCESS 


Research of curreht directives showed that TEE facility 
missions were not clearly defined and thus some overlap 
existed. NAVMATINST 5450,27 acknowledges the existence of 


overlap and encourages a limited amount of competition. 


Although the Project Manager is granted a great amount 
of authority and responsibility by a Project Charter, few 
procedures seem to be ciearly defined in the T&E facility 
selection process. Current directives seem to stimulate a 
buyer-seller relationship between the PM and the TEE 


facility. 


Previous attempts, such as the establishement of TECO, 
have been unsuccessfull in providing fOr overall 
coordination of TEE within NAVAIR. The establishment of 
AIR-96 15 an attempt to strengthen this coordination, but 
its effectiveness remains to be tested. There appears to be 
a problem of overlapping responsibilities in the control of 
resources inherent in the proposed systen due to tne 


double-hatting of 067 апа O06R as pointed out in Chapter III. 


Dawe ee TY ORGANIZATION 


рее for an Organization to be effective it must be 


responsive and adaptable Eo ite changing external 
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environment. This is particularly true of TEE facilities 
which are so intimately involved with a rapidly changing 
technology. The stagnation that has hampered operations at 
the Point Mugu complex must be set aside. The PTC 
organization is structured to avoid stagnancy and to be 
reactive to its users! needs. However, an organization can 
only be as effective as the people working for it. Only 
time can determine if the personnel are willing to 


participate in its dynamic development. 


ЕН ЕКЕНСІЗ ОК FUNDING METHODS 


The authors had the impression that the primary driving 
force behind a PM's selection of a T&E facility was cost. 
Research into the means by which activities receive their 
funds revealed that there are two major financial management 
techniques used by acilities. They are NIF and R&S. 
Additionally, the major TSE facilities, as specified in 
DODDIR 3200,11, have procedures and regulations governing 
their funding as a result of being MRTFB wembers. Іп 
general, then, a T&E facility is either under NIP or RMS for 
its financial management and receives its funds either from 
oO MUDESAZInstitutional Funds. 


NIF appears to be a more dynamic managerial nethod thaa 
RHS.  NIF requires an activity to receive its operating 
funds through the Sale of goods and services to customers, 
This causes several econouic consequences: formation of 2 


buyer-seller relationship а POP customners. a neei t 
у Pr t 


м о 


Е 
ensure their satisfaction, and the formulation of 


7 


competitive environment. RMS, іп contrast to NIF, onl 


Lo 


encourages a faciltiy to efficiently use its resources. А 
RMS activity is thus guaranteed of a user for its services 
and does not need to seek customers or ensure their 


Satisfaction. 
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The difference in funding policy causes different 


charges to the user as discussed in Chapter IV. 


D. PROJECT MANAGERS! PARTICIPATION 


Respones from the questionnaire indicated that 
technical capabilities an installation can offer for a 
project was the most important criterion used by decision 
makers in selecting a T&E facility. In contrast to the 
authors! original assumption, cost estimates were not number 
one but a distant third after technical capability and 


personnel expertise, 


A Project Manager has considerable latitude іп 
selecting а ТСЕ facility and is not normally hampered by 
external constraints such as project urgency or higher 
mublonprtwuprior Selection. Ав was expected, he could be 
limited in his selection process by technical constraints 
Ворот, one facility; and, approximately 45 percent of 
the PMS were. In addition to the latitude а РИ enjoys 
during initial faciiity selection, he can, later on, alter 


the selection to another facility. 


Tne process used by Project Managers to learn of an 


installation's capability was based heavily upon past 
experience and associazes. Next in importance was the 
liaison offices established in Washington. Тһе liaisons 


жакет ра паре: Ву Of the Pris, but the services could 
be improved. Решено № indicated that some iiaisoa 
offices were better than others, which would have a direct 
influence on a Project Manager's perception of tnat 
facility- Publications were not considered а useful 


information source by the managers. 


Project Managers do not desire à competitive 
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environment in TSE. They believe the cost of TEE would be 
reduced if competition among facilities were eliminated. 
піс Ше іс іп conflict with NAVMATINST 5450.27 which 
expressed the opinion that limited competition is both 


necessary and good for TEBE. 


Responses from the questionnaire indicated a clear 
preference for NWC and the services it offers over other 
comparable facilities. This high degree of satisfaction 
with NWC was held by both Project Managers and Project 
Officers. It would be to the benifit of other facilities to 
investigate this finding further and determine the reasons 


behind the NWC number one ranking. 


E. DINMESDOOECTIMEOSRLECTIION PROCESS 


The facility selection process developed in Chapter Vi 
provides a systematic means by which a PS could determine 
Which facility is most technically capable of handling his 
Protec Ihe 26 Lunctionel areas derined by PNTC personnel 
seen to exhaust the capabilities of a T&E facility in terns 
of services, Although it is recognized that the point 
totalization could be hand calculated, the computer progran 


aids greatly in a sensitivity analysis of weighting factors. 


There exists a need to make an objective determination 
of a facility's rating in each of the 26 areas aná a method 
of updating it. This could pe effectively accomplished by an 


organization such as AIR-U6. 


Additional research could be done in developing a more 
objective and easily understandable method of assigning 
Ее тонет  Еасвог5, This might be done by a form 
which asks specific  4uestions concerning the number of 


Hoc О ре [азо ıncejdeut to a project etc. and 





asSigning point values accordingly. 


It. must be emphasized that the process is based on 
technical capability, only, and does not address itself to 
other factors such as cost which must be eventually 


considered. 





Appendix A 


PMR Financiai Budget FY 74-81 


(in milłion o£ dollars) 


Operations 


Institutional 
Direct Charges 

Range Improvements 
mistitucional 
Others 


- 


TOTALS 


Operations 
Institutional 
Direct Charges 

Range Improvements 
Institutional 
Others 


TOTALS 


74 


732427 


262 


u) 


96.84 


79 


20293 


206 


75 


68.70 


20559 


104.08 


79 


88.35 


20.76 


140 


76 


Тарас 


11.64 


119 299 


80 


88.72 


ОИС 


AS 


1.08 


129.48 


ЁТ 


69.46 


20.76 


20.268 


77 


jd = 





PMR Direct Charges 


(Users 


Fundin 


(thousands of dollars) 


Navy 


RDTSE,N 
ОМ, М 
NIF 


RDT&EE,AF 
ОБН, АР 
ОБМ, А 


20,550 _ 


Projected Total Operations 


Range User CY 775 
Navy 14,293 
NASA 32 
Civilian 0 
DoD 12 
ДЕС n 
USAF 242 
Агпу D 


БОРАП 15,588 


ПЕС 


12,847 
32 

0 

12 

76 

896 
172 


14,035 
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127937 
32 

0 

12 

76 

885 
172 


13,714 


g) 


17,500 
300 
3,400 
13,800 
900 
300 
525 
75 
800 
440 
360 
150 
19,350 

СҮ 78 

12,651 

32 

0 

12 

76 

837 

232 

13,840 





Naval Missile Center 
(FY 75) 


Customer kork Requests 
(dollar value) 


NAVAIR $94,674, 383 
NAVSEA | ИЗ, 584 
NAVELEC 21132 21526 
CNR $252,065 
Other Navy So, OF 7,418 
Marines $211,349 
Air Force $3,389,420 
Army SI 39, 714 
NASA $39,692 
Other Govt. Dept. $294,997 
Private Parties $23,630 
ASO | $28,936 
NAVILCO $4,242,761 


TOTALS Сот 830.075 
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APPENDIX B 


TEST AND EVALUATION ASSESSMENT 


Please enter the most appropriate number in the box at 
che richt of each statement. 
2 Seroneizuremee 2.-Eaeree 9.-.0o Opinion 4.-Disagree 5.-Strongly Disarree 








i. More than one Test and Evaluation (T&E) facility was 
capable of providing the similiar services needed for 
my project. 

2. The technical constraints of my project limited selection 
to one TEE facility. 

2. The urgency of my project limited selection to one T&E 
facility. 

4, Current publications provide a useful source of 
шотта ол сп the capabilities of T&E facilities. 

5. Other project managers provide a useful source of 
roroa ron on Che capabilities of T&E facilities. 

6. Liaisons from T&E facilities provide a useful source 
or jinl ormation on the capabilities of their facilities. 

7. A project manager's previous assismments provide a 
valuable source of information on the capabilities 
ОР TSE facilities. | 

B. Staff assistance within & system command is useful 
ое ие the capabilities of T&E facilities. 

ес unif$cuntedifference in costs to projects 
from Tah facilities offering comparable services. 


10. Different funding techniques (NIF/RMS,etc.) used by 


GJ OU J OUUU U 


T&E facilities tend to mask the actual T&E cost of 


& project. 
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Please enter the most appropriate number in the box 
at the right of each statement. 
1.=Stronely Azree 2.=Arree 3.=No Opinion 4.=Disarree 5.=Stronrly Disarree 





оное a lar facility has been chosen for a project a 
project manager has no latitude in altering the L| 
selection. 


HERE FOr tne Tollowing 3 questions comparable services is 
meant to be those services which more than one T&E facility 
ig capable of providing in support of a project. 


12. As regards to facilities offering comparable services, 
there is a significant difference in the information 
feedback provided to a project manager to effectively 
monitor tmesprogress of his project. 

шие .S a slenii icant difference in the technical 
CAPS tise sommes facilities offering comparable services. 

14. As regards to facilities offering comparable services, 


Pheresiszssyen)ticent Glfference In the ease of obtaining 





non-routine information on the progress of a project. 


15% Eunding level for TSE facilities! modernization is 


J LIU LJ 


adequate to support my project needs. 
16. Hank each of the following factors with respect to its 
importance in the facility selection process. 


Cost Estimates 
Personnel Expertise 
Technical Capability 
Logistics 

Program Cooperation 
dest ScCheduring 
Others (specify) 


ч 
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Please enter the most appropriate number in the box at the 
right of each statement. 
e=Stronzly Agree 2.=Agree 3.=No Opinion 4.=Disarree 5.=Stronrly Disarree 


17. Rank the following T&E facilities in their overall 








effectiveness in providing satisfactory services to 
project managers. 
1. Most effective ------- &. Least effective 


А. Naval Weapon Center, China Lake 

B. Pacific Missile Test Center, Pt. Mugu 

С. Naval Air Test Center, Patuxent River 

D. Armament Development and Test Center, 
Eglin AFB 

E. White Sands Missile Rance 

F. U.S. Army Yuma Proving rounds 

С. Altantic Fleet Weapons Range 

GSAT Perce Eastern Test Range, Patrick AFB 


18. Increased competition among T&E facilities would 
increase the effectiveness of services provided to 
project managers. 

19. Competition among T&E facilities should be eliminated 
ООШОТ Пет ТО reduce the additional. cost of overlapping 
capabilities. 

20. Selection of a contractor influences the selection 
of a T&E facility. 

21. DCP's are overly restrictive in their testing requirements 
Апа ече int tne project manager's selection to one 
toe facility: 


22. Weapon systems T&E requirements are identified carly 


QDD UDO O0 OG 000900099 


eno Л O рел E onding justification for facility 


modernization. 
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Wnen answering the following questions use the appropriate 


numerical response as indicated below. 


1.- 
23. 


25. 


25+ 


26. 


27. 


сё, 
29. 


30. 
21. 


All 2.= Most 3.= Some 4.= Few 5.= None 

Шо гас т Ес аге Солвізселс іп their method of 
accenn project priorities for completion. 

Шу гас 1 1е5 Show a renuine interest in providing 
ргосгратеспррого for successful completion of the 
project. 

Pefreyemwieszmoteazpy the P.M. in a project's 
progress are examined and corrected by the T&E 
facilities. 

Teri acili ties Capable of providing services for my 
project have shown eagserness for my project. 
MERAC ceS actively participate in the selection 
process of weapon system test assignment. 

What is your rank/GS level? | 


Highest level formal education achieved. 


Associate Baccalaureate Master(technical) 


tlaster(non-technical) Doctorate 
Number of years involved in '"'éE assignments. 
DoS oude sire results of this questionnaire? 


YES NO 


a EE RE 





Mailing Adaress 


form number 
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